From jay.krell at cornell.edu Mon Jun 1 00:15:26 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:15:26 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B4BE9.1000902@marino.st> References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. - Jay On May 31, 2015, at 10:59 AM, John Marino wrote: > I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > Should I put it back? From the last response I'm guessing that wasn't > the problem. > > John > > > On 5/31/2015 19:09, Jay K wrote: >> Oh probably the problem is using the backend premature actually. >> Darn, I can't help right now. >> >> - Jay >> >> >> >> ------------------------------------------------------------------------ >> From: jay.krell at cornell.edu >> To: adacore at marino.st; m3devel at elegosoft.com >> Date: Sun, 31 May 2015 16:45:42 +0000 >> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >> (0x100), expected 0x110 >> >> Hm. Strange. I need to read more closely. I see it skipping shipping, >> but my read was it didn't try that. >> So.. with INSTALL_CM3_IN_BIN=yes? >> I need to try this all, and "fix" it to set that. The whole thing in >> m3cc/src/m3makefile is relatively >> new vs. when I was actively using all this. :( >> >> - Jay >> >> >>> Date: Sun, 31 May 2015 16:57:02 +0200 >>> From: adacore at marino.st >>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >> (0x100), expected 0x110 >>> >>> On 5/31/2015 11:53, Jay K wrote: >>>> John, have you tried make-dist.py? >>>> i.e. >> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>> >>>> While it might not be exactly what you need, it should demonstrate the >>>> elements. >>>> >>>> I will try to try this all soon, really, I hope so. >>>> >>>> It starts with a working compiler, does no in-place updates, build into >>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>> It does "min" and "all". >>>> >>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>> >>>> - Jay >>> >>> Hi Jay, >>> I'm getting the same error as always using make-dist script: >>> http://leaf.dragonflybsd.org/~marino/m3d.log >>> >>> Thanks, >>> John From jay.krell at cornell.edu Mon Jun 1 00:47:42 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:47:42 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> Ok,caveat that I haven't tried this, could be more waste of your time,but in scripts/pkginfo.txt try moving the m3cc line down to just after cm3. This way, if the config is looking all around for it, as was mentioned and as we should avoid, it won't find it prematurely. I was never quite sure what order to put this entry, as it has in a sense no edges to or from it in the build dependency graph. But there is the sensitivity as to when to use it, so when to ship it. Shipping after cm3 should avoid using it prematurely. - Jay On May 31, 2015, at 3:15 PM, Jay wrote: > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > - Jay > > On May 31, 2015, at 10:59 AM, John Marino wrote: > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >> Should I put it back? From the last response I'm guessing that wasn't >> the problem. >> >> John >> >> >> On 5/31/2015 19:09, Jay K wrote: >>> Oh probably the problem is using the backend premature actually. >>> Darn, I can't help right now. >>> >>> - Jay >>> >>> >>> >>> ------------------------------------------------------------------------ >>> From: jay.krell at cornell.edu >>> To: adacore at marino.st; m3devel at elegosoft.com >>> Date: Sun, 31 May 2015 16:45:42 +0000 >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>> >>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>> but my read was it didn't try that. >>> So.. with INSTALL_CM3_IN_BIN=yes? >>> I need to try this all, and "fix" it to set that. The whole thing in >>> m3cc/src/m3makefile is relatively >>> new vs. when I was actively using all this. :( >>> >>> - Jay >>> >>> >>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>> From: adacore at marino.st >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>>> >>>> On 5/31/2015 11:53, Jay K wrote: >>>>> John, have you tried make-dist.py? >>>>> i.e. >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>> >>>>> While it might not be exactly what you need, it should demonstrate the >>>>> elements. >>>>> >>>>> I will try to try this all soon, really, I hope so. >>>>> >>>>> It starts with a working compiler, does no in-place updates, build into >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>> It does "min" and "all". >>>>> >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>> >>>>> - Jay >>>> >>>> Hi Jay, >>>> I'm getting the same error as always using make-dist script: >>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>> >>>> Thanks, >>>> John From jay.krell at cornell.edu Mon Jun 1 00:51:25 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:51:25 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> Message-ID: <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> Ps. Update pkginfo.txt and then try make-dist.py - Jay On May 31, 2015, at 3:47 PM, Jay wrote: > Ok,caveat that I haven't tried this, could be more waste of your time,but in scripts/pkginfo.txt try moving the m3cc line down to just after cm3. This way, if the config is looking all around for it, as was mentioned and as we should avoid, it won't find it prematurely. > > > I was never quite sure what order to put this entry, as it has in a sense no edges to or from it in the build dependency graph. But there is the sensitivity as to when to use it, so when to ship it. Shipping after cm3 should avoid using it prematurely. > > - Jay > > On May 31, 2015, at 3:15 PM, Jay wrote: > >> I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. >> >> - Jay >> >> On May 31, 2015, at 10:59 AM, John Marino wrote: >> >>> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >>> Should I put it back? From the last response I'm guessing that wasn't >>> the problem. >>> >>> John >>> >>> >>> On 5/31/2015 19:09, Jay K wrote: >>>> Oh probably the problem is using the backend premature actually. >>>> Darn, I can't help right now. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> From: jay.krell at cornell.edu >>>> To: adacore at marino.st; m3devel at elegosoft.com >>>> Date: Sun, 31 May 2015 16:45:42 +0000 >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>>> (0x100), expected 0x110 >>>> >>>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>>> but my read was it didn't try that. >>>> So.. with INSTALL_CM3_IN_BIN=yes? >>>> I need to try this all, and "fix" it to set that. The whole thing in >>>> m3cc/src/m3makefile is relatively >>>> new vs. when I was actively using all this. :( >>>> >>>> - Jay >>>> >>>> >>>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>>> From: adacore at marino.st >>>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>>> (0x100), expected 0x110 >>>>> >>>>> On 5/31/2015 11:53, Jay K wrote: >>>>>> John, have you tried make-dist.py? >>>>>> i.e. >>>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>>> >>>>>> While it might not be exactly what you need, it should demonstrate the >>>>>> elements. >>>>>> >>>>>> I will try to try this all soon, really, I hope so. >>>>>> >>>>>> It starts with a working compiler, does no in-place updates, build into >>>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>>> It does "min" and "all". >>>>>> >>>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>>> >>>>>> - Jay >>>>> >>>>> Hi Jay, >>>>> I'm getting the same error as always using make-dist script: >>>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>>> >>>>> Thanks, >>>>> John From hendrik at topoi.pooq.com Mon Jun 1 02:29:44 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 31 May 2015 20:29:44 -0400 Subject: [M3devel] User interfaces for Modula 3. Message-ID: <20150601002944.GA10751@topoi.pooq.com> What user interface libraries are available for Modula 3? I know there's Trestle. But is there something like GTK or QT or somethng I can use like cairo to draw really pretty text? Anything that supports lots of Unicode? I don't mind havein to do my own character placement based on font metrics of some sort; not now, bit later, I may have some really weird layout constraints. In case you're wodering about the application: I'm doing preliminary planning for something like a text editor with several windows, one with the raw text, and another with a continuous (as continuous as I have cpu cycles for) display of the results of rather complex calculations on that text (such as proof checking or described graphics). Modula 3 isn't the only candidate for a programming language, just the leading candidate for historical and bootstrapping reasons. The others at the moment are OCAML and some kind of statically typed Scheme. And of course, I may decide that theo whole projecct is infeasible. -- hendrik From rodney_bates at lcwb.coop Mon Jun 1 05:28:13 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 31 May 2015 22:28:13 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: <556BD14D.5050507@lcwb.coop> I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes to fail regardless of the order of building the two. OTOH, if cm3 were built and shipped, then m3cc came immediately after, it should work, because m3cc contains no Modula-3 code, and the new cm3 would not, when building it, run cm3cg. After that, there would be a consistent set. for further compiles. But any package containing M3 code built between cm3 and m3cc would fail. On 05/31/2015 05:15 PM, Jay wrote: > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > - Jay > > On May 31, 2015, at 10:59 AM, John Marino wrote: > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >> Should I put it back? From the last response I'm guessing that wasn't >> the problem. >> >> John >> >> >> On 5/31/2015 19:09, Jay K wrote: >>> Oh probably the problem is using the backend premature actually. >>> Darn, I can't help right now. >>> >>> - Jay >>> >>> >>> >>> ------------------------------------------------------------------------ >>> From: jay.krell at cornell.edu >>> To: adacore at marino.st; m3devel at elegosoft.com >>> Date: Sun, 31 May 2015 16:45:42 +0000 >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>> >>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>> but my read was it didn't try that. >>> So.. with INSTALL_CM3_IN_BIN=yes? >>> I need to try this all, and "fix" it to set that. The whole thing in >>> m3cc/src/m3makefile is relatively >>> new vs. when I was actively using all this. :( >>> >>> - Jay >>> >>> >>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>> From: adacore at marino.st >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>>> >>>> On 5/31/2015 11:53, Jay K wrote: >>>>> John, have you tried make-dist.py? >>>>> i.e. >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>> >>>>> While it might not be exactly what you need, it should demonstrate the >>>>> elements. >>>>> >>>>> I will try to try this all soon, really, I hope so. >>>>> >>>>> It starts with a working compiler, does no in-place updates, build into >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>> It does "min" and "all". >>>>> >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>> >>>>> - Jay >>>> >>>> Hi Jay, >>>> I'm getting the same error as always using make-dist script: >>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>> >>>> Thanks, >>>> John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 05:32:29 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 31 May 2015 22:32:29 -0500 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <20150601002944.GA10751@topoi.pooq.com> References: <20150601002944.GA10751@topoi.pooq.com> Message-ID: <556BD24D.7010902@lcwb.coop> There is a QT binding in cm3/m3-ui/qt. I have no experience with it. On 05/31/2015 07:29 PM, Hendrik Boom wrote: > What user interface libraries are available for Modula 3? > > I know there's Trestle. > > But is there something like GTK or QT or somethng I can use like cairo > to draw really pretty text? Anything that supports lots of Unicode? > I don't mind havein to do my own character placement based on font > metrics of some sort; not now, bit later, I may have some really > weird layout constraints. > > In case you're wodering about the application: > > I'm doing preliminary planning for something like a text editor with > several windows, one with the raw text, and another with a continuous > (as continuous as I have cpu cycles for) display of the results of rather > complex calculations on that text (such as proof checking or > described graphics). > > Modula 3 isn't the only candidate for a programming language, just the > leading candidate for historical and bootstrapping reasons. The others > at the moment are OCAML and some kind of statically typed Scheme. > > And of course, I may decide that theo whole projecct is infeasible. > > -- hendrik > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 08:40:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:40:38 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556BD14D.5050507@lcwb.coop> References: <556700B0.8060500@marino.st> <5567DA8C.3050802@lcwb.coop>,<556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , <556BD14D.5050507@lcwb.coop> Message-ID: Yes, exactly, that's what I was trying to say, in my later/last mail. Yes the compability is definitely two-way. There is a one to one mapping back and forth. cm3 goes with cm3cg; cm3cg goes with cm3. Neither is compatible with older nor necessarily newer versions. They would be statically linked together if not for the GPL. When I first placed m3cc in pkginfo.txt years ago, I was only thinking of build order and, like, "override". m3cc imports nothing and nothing imports m3cc. In the Modula-3 module/library/interface sense. It can be built when you have nothing. So I figured put it nearly first. Only now today did I consider the shipping order. It should be shipped adjacent to cm3, either right before or right after. If you split build and ship, then it can still be built early, or at almost any time. But we have one ordering for build and ship, so let the shipping order decide. I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. I still think it broke upgrade.py. I can workaround. But I'd like to remove the environment variable check. There are many order dependencies in build and ship. This is just one. - Jay > Date: Sun, 31 May 2015 22:28:13 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; adacore at marino.st > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > to fail regardless of the order of building the two. > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > it should work, because m3cc contains no Modula-3 code, and the new > cm3 would not, when building it, run cm3cg. After that, there would > be a consistent set. for further compiles. But any package containing > M3 code built between cm3 and m3cc would fail. > > On 05/31/2015 05:15 PM, Jay wrote: > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > - Jay > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > >> Should I put it back? From the last response I'm guessing that wasn't > >> the problem. > >> > >> John > >> > >> > >> On 5/31/2015 19:09, Jay K wrote: > >>> Oh probably the problem is using the backend premature actually. > >>> Darn, I can't help right now. > >>> > >>> - Jay > >>> > >>> > >>> > >>> ------------------------------------------------------------------------ > >>> From: jay.krell at cornell.edu > >>> To: adacore at marino.st; m3devel at elegosoft.com > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >>> (0x100), expected 0x110 > >>> > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > >>> but my read was it didn't try that. > >>> So.. with INSTALL_CM3_IN_BIN=yes? > >>> I need to try this all, and "fix" it to set that. The whole thing in > >>> m3cc/src/m3makefile is relatively > >>> new vs. when I was actively using all this. :( > >>> > >>> - Jay > >>> > >>> > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > >>>> From: adacore at marino.st > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >>> (0x100), expected 0x110 > >>>> > >>>> On 5/31/2015 11:53, Jay K wrote: > >>>>> John, have you tried make-dist.py? > >>>>> i.e. > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > >>>>> > >>>>> While it might not be exactly what you need, it should demonstrate the > >>>>> elements. > >>>>> > >>>>> I will try to try this all soon, really, I hope so. > >>>>> > >>>>> It starts with a working compiler, does no in-place updates, build into > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > >>>>> It does "min" and "all". > >>>>> > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > >>>>> > >>>>> - Jay > >>>> > >>>> Hi Jay, > >>>> I'm getting the same error as always using make-dist script: > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > >>>> > >>>> Thanks, > >>>> John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 08:54:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:54:29 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B5356.40809@lcwb.coop> References: <556700B0.8060500@marino.st>, <5567DA8C.3050802@lcwb.coop>,,<556817A1.2090206@marino.st>, ,,<20150529105451.e9c9b9c2a5578a28bc654f97@elego.de>, ,,<55682D1A.2030504@marino.st>, ,,<20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de>, ,,<55683689.2090709@marino.st>, ,,<20150529122732.1396eb079ff9b404bcb21e40@elego.de>, ,,<5568421D.3050209@marino.st>, ,,<20150529155833.8454664b88391d93708e8f85@elego.de>, ,,<55687266.3090400@marino.st>, ,,<20150529162028.388911007a614006049beb42@elego.de>, ,,<20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, ,,<556885FD.6010800@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, ,,<55689065.1040207@marino.st>, ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, ,,, , , <556B213E.8030108@marino.st>, , , <556B4BE9.1000902@marino .st>,<556B5356.40809@lcwb.coop> Message-ID: It only happens if you ship. Shipping in the wrong order can always break things. You cannot ship in arbitrary order and incorrect order can cause arbitrary problems. That doesn't mean we should never ship and always have a separate copy mechanism. The knowledge of what to ship is meant to be in the m3makefiles. Not also encoded elsewhere. The environment variable check really should be removed. Perhaps perhaps perh aps shipping order can be enforced? For things other than m3cc, this seems possible. Immediate matching dependencies should be present in the destination. Using the information that is already present in the system as to immediate and matching, in terms of Modula-3 imports/interfaces. m3cc is unique in not fitting in that however. As well, the dependency between m3cc/cm3 -- what order would you say it is? Circular but the tie can be broken arbitrarily? Introduce a command that can ship multiple packages at once? I kind of want that anyway, but in this case you'd arrive back at the environment variable, as long as cm3 has it..though cm3 only needs it for NT and hypothetical other targets like AIX/OS2/DOS/DJGPP. However -- how about this suggestion -- make the environment variable check for relative native target? Only skip cm3 shipping when we know it doesn't work, I..e NT/Cygwin/Mingwin? I like this. It is a special case either way, but at least we can demonstrate a cleaner system on most systems.. - Jay > Date: Sun, 31 May 2015 13:30:46 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; adacore at marino.st > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 12:59 PM, John Marino wrote: > > I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > Should I put it back? From the last response I'm guessing that wasn't > > the problem. > > > > John > > > > It apparently wasn't the only problem, but it definitely was a problem. > Don't put it back. I causes the new cm3cg executable to be copied > to your bin directory immediately after it has been built. It is > incompatible with the old cm3 executable, and the next attempted > build (m3core, as I remember) runs the old cm3, then the new cm3cg, > which correctly recognizes that its input is in an old version format > and refuses, giving the error message. > > > > > On 5/31/2015 19:09, Jay K wrote: > >> Oh probably the problem is using the backend premature actually. > >> Darn, I can't help right now. > >> > >> - Jay > >> > >> > >> > >> ------------------------------------------------------------------------ > >> From: jay.krell at cornell.edu > >> To: adacore at marino.st; m3devel at elegosoft.com > >> Date: Sun, 31 May 2015 16:45:42 +0000 > >> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >> (0x100), expected 0x110 > >> > >> Hm. Strange. I need to read more closely. I see it skipping shipping, > >> but my read was it didn't try that. > >> So.. with INSTALL_CM3_IN_BIN=yes? > >> I need to try this all, and "fix" it to set that. The whole thing in > >> m3cc/src/m3makefile is relatively > >> new vs. when I was actively using all this. :( > >> > >> - Jay > >> > >> > >>> Date: Sun, 31 May 2015 16:57:02 +0200 > >>> From: adacore at marino.st > >>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >> (0x100), expected 0x110 > >>> > >>> On 5/31/2015 11:53, Jay K wrote: > >>>> John, have you tried make-dist.py? > >>>> i.e. > >> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > >>>> > >>>> While it might not be exactly what you need, it should demonstrate the > >>>> elements. > >>>> > >>>> I will try to try this all soon, really, I hope so. > >>>> > >>>> It starts with a working compiler, does no in-place updates, build into > >>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > >>>> It does "min" and "all". > >>>> > >>>> If you set the STAGE environment variable, it uses that instead of /tmp. > >>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > >>>> > >>>> - Jay > >>>> > >>> > >>> Hi Jay, > >>> I'm getting the same error as always using make-dist script: > >>> http://leaf.dragonflybsd.org/~marino/m3d.log > >>> > >>> Thanks, > >>> John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 08:48:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:48:12 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B5D6A.3030403@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B5D6A.3030403@lcwb.coop> Message-ID: > was looking several alternative places for an executable cm3cg I agree it did/does do that. I did put that in. I thought it was a good idea at the time. I realize now it is a mistake. If that behavior is still present, it will be advisable also to first update one's config to stop doing that. Basically, "ship" is a very carefully ordered operation, the shipping of multiple packages. If you have no bugs, and you ship in the correct order, then you don't need DESTDIR. IF you have no bugs, but you ship in the incorrect order, you cause a bad mixing of things. If you use "DESTDIR" then you can ship safely in any order and do some switch at the end, such as by reseting PATH to point to the new set of files. If you have circular dependencies, there might not be a workable order. I agree this could be the case. make-dist.py is written to avoid but might be slightly wrong. I suspect I only ever ran make-dist after upgrade. I have started setting up VMs..though NetBSD, FreeBSD, OpenBSD all fail under VirtualBox if Hyper-V is enabled...but at least Debian is making progress. So hopefully I can test and fix this properly soon. And then I'll back to using the C backend anyway. :) - Jay > Date: Sun, 31 May 2015 14:13:46 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > The error message you keep getting pretty certainly means it is running a > new cm3cg with an old cm3. And it consistently happens immediately after > cm3cg is rebuilt. > > The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > your failed builds, and I don't think that package would ever run cm3cg, as > there is no Modula-3 code in it. Which means there is no proof that the > version of cm3cg that would have been used before it was rebuilt is the old > one. (Could it be a new one left over from a previous incomplete build?) > > One experiment that would prove this would be, in the state your are > starting from before trying a full build, try any means of running > cm3 on a package that contains Modula-3 code, to see if it avoids the > recurring problem. > > But I have another theory. I don't know what this ports bootstrap > is, but there was a time when things where set up so that cm3, instead > of looking in just the same directory as its own executable was found, > was looking several alternative places for an executable cm3cg. This > caused the same problem to occur in a different way. Even when the > newly rebuilt cm3cg executable was not copied to the bin directory > right away, its mere existence where it was built was causing it to > be found the next time cm3 started up, leading to the same failure. > > I don't remember how this was fixed, but either Jay or I did something > about it. Perhaps the ports bootstrap was made during a time when this > behavior was happening. > > On 05/31/2015 02:41 AM, John Marino wrote: > > On 5/29/2015 20:43, John Marino wrote: > >> On 5/29/2015 20:15, Olaf Wagner wrote: > >>> > >>> Well, yes, I understand that. I would have tried your exact setup, > >>> but I have no system available to test that on. > >>> > >>> At least I have validated that based on the origianl 5.8.6 installation > >>> archive for AMD64_FREEBSD you can build the new compiler from the current > >>> sources with a simple call of the upgrade.sh script. which I still doubted > >>> yesterday. > >> > >> > >> The card I still have left to play is to extract the bootstrap, let it > >> overwrite itself per Rodney's technique and then build the real compiler > >> (dumping the whole "intermediate" area). > > > > FWIW, this did not work either. Rodney's technique doesn't seem to work > > with the ports bootstrap. I don't know why it would work with provided > > 5.8.6 but not rebuilt one. As far as I can tell, I did what he said > > would always work. > > > > I'm at a loss as to where to go from here. It's starting to look like I > > have to throw away the ports 5.8.6 bootstrap and start over somehow. I > > don't think it should be this hard. :( > > > > John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 09:05:31 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:05:31 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B743E.7030504@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B5D6A.3030403@lcwb.coop>, <556B6075.6010703@marino.st>, <556B743E.7030504@lcwb.coop> Message-ID: > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > 274: foreach root in [ m3cc, bin ] > to: > 274: foreach root in [ bin ] > Before building the Modula-3 head. Yes, agreed. I still want the environment variable check removed, but this is another problem, that I caused, and I did fix long ago, but is causing grief still, sorry about that. Here, I fixed it in 2010: https://github.com/modula3/cm3/commit/65551f30a6c3ed2cd9740394a9082ffab1b07faa INSTALL_ROOT/bin/cm3cg for native builds ROOT/m3-sys/m3cc/HOST/TARGET/cm3cg for cross builds "Later" (never?) we can think about "installed" cross builds. In particular, when I make m3cg interface changes, I get burned by reaching in for ROOT/m3-sys/m3cc/x/cm3cg. The code is a lot smaller now and more correct. Sorry about that. I was being ambitious in trying to make things work, and it turned out to be quite incorrect once things are better understood. - Jay > Date: Sun, 31 May 2015 15:51:10 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 02:26 PM, John Marino wrote: > > On 5/31/2015 21:13, Rodney M. Bates wrote: > >> The error message you keep getting pretty certainly means it is running a > >> new cm3cg with an old cm3. And it consistently happens immediately after > >> cm3cg is rebuilt. > >> > >> The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > >> your failed builds, and I don't think that package would ever run cm3cg, as > >> there is no Modula-3 code in it. Which means there is no proof that the > >> version of cm3cg that would have been used before it was rebuilt is the old > >> one. (Could it be a new one left over from a previous incomplete build?) > > > > That's not possible. The bootstrap is packaged in a compressed tarball > > and extracted for the sole purpose of building M3. It would have not > > leftovers in it. > > > > It comes with cm3, cm3cg, m3bundle, and mklib in the bin directory. > > > > the contents of the bootstrap M3 cm3.cfg are: > > INSTALL_ROOT = path() & "/.." > > include (path() & "/../etc/modula3/AMD64_FREEBSD") > > > > > >> One experiment that would prove this would be, in the state your are > >> starting from before trying a full build, try any means of running > >> cm3 on a package that contains Modula-3 code, to see if it avoids the > >> recurring problem. > > > > I think this is N/A. The bootstrap compiler is extracted to a directory > > that is not on the path (and only exists right before trying to build > > modula3 port) > > > > > >> But I have another theory. I don't know what this ports bootstrap > >> is, but there was a time when things where set up so that cm3, instead > >> of looking in just the same directory as its own executable was found, > >> was looking several alternative places for an executable cm3cg. This > > > > It's 5.8.6 and the cm3.cfg is listed above. You can download it > > yourself here: > > http://downloads.dragonlace.net/m3/m3-bootstrap.AMD64.FREEBSD.92.tar.bz2 > > > > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > > 274: foreach root in [ m3cc, bin ] > > to: > > 274: foreach root in [ bin ] > > Before building the Modula-3 head. > > > > >> caused the same problem to occur in a different way. Even when the > >> newly rebuilt cm3cg executable was not copied to the bin directory > >> right away, its mere existence where it was built was causing it to > >> be found the next time cm3 started up, leading to the same failure. > >> > >> I don't remember how this was fixed, but either Jay or I did something > >> about it. Perhaps the ports bootstrap was made during a time when this > >> behavior was happening. > > > > Probably not as I am 99% sure the bootstrap was built from the release > > source. > > > > Thanks, > > John > > > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 09:07:25 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 09:07:25 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> References: <556700B0.8060500@marino.st> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> Message-ID: <556C04AD.5000509@marino.st> On 6/1/2015 00:51, Jay wrote: > Ps. Update pkginfo.txt and then try make-dist.py > Okay, I applied this patch: http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt and the same thing happened, only later: http://leaf.dragonflybsd.org/~marino/m3e.log John From jay.krell at cornell.edu Mon Jun 1 09:05:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:05:05 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B743E.7030504@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B5D6A.3030403@lcwb.coop>, <556B6075.6010703@marino.st>, <556B743E.7030504@lcwb.coop> Message-ID: > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > 274: foreach root in [ m3cc, bin ] > to: > 274: foreach root in [ bin ] > Before building the Modula-3 head. Yes, agreed. I still want the environment variable check removed, but this is another problem, that I caused, and I did fix long ago, but is causing grief still, sorry about that. Here, I fixed it in 2010: https://github.com/modula3/cm3/commit/65551f30a6c3ed2cd9740394a9082ffab1b07faa INSTALL_ROOT/bin/cm3cg for native builds ROOT/m3-sys/m3cc/HOST/TARGET/cm3cg for cross builds "Later" (never?) we can think about "installed" cross builds. In particular, when I make m3cg interface changes, I get burned by reaching in for ROOT/m3-sys/m3cc/x/cm3cg. The code is a lot smaller now and more correct. Sorry about that. I was being ambitious in trying to make things work, and it turned out to be quite incorrect once things are better understood. - Jay > Date: Sun, 31 May 2015 15:51:10 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 02:26 PM, John Marino wrote: > > On 5/31/2015 21:13, Rodney M. Bates wrote: > >> The error message you keep getting pretty certainly means it is running a > >> new cm3cg with an old cm3. And it consistently happens immediately after > >> cm3cg is rebuilt. > >> > >> The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > >> your failed builds, and I don't think that package would ever run cm3cg, as > >> there is no Modula-3 code in it. Which means there is no proof that the > >> version of cm3cg that would have been used before it was rebuilt is the old > >> one. (Could it be a new one left over from a previous incomplete build?) > > > > That's not possible. The bootstrap is packaged in a compressed tarball > > and extracted for the sole purpose of building M3. It would have not > > leftovers in it. > > > > It comes with cm3, cm3cg, m3bundle, and mklib in the bin directory. > > > > the contents of the bootstrap M3 cm3.cfg are: > > INSTALL_ROOT = path() & "/.." > > include (path() & "/../etc/modula3/AMD64_FREEBSD") > > > > > >> One experiment that would prove this would be, in the state your are > >> starting from before trying a full build, try any means of running > >> cm3 on a package that contains Modula-3 code, to see if it avoids the > >> recurring problem. > > > > I think this is N/A. The bootstrap compiler is extracted to a directory > > that is not on the path (and only exists right before trying to build > > modula3 port) > > > > > >> But I have another theory. I don't know what this ports bootstrap > >> is, but there was a time when things where set up so that cm3, instead > >> of looking in just the same directory as its own executable was found, > >> was looking several alternative places for an executable cm3cg. This > > > > It's 5.8.6 and the cm3.cfg is listed above. You can download it > > yourself here: > > http://downloads.dragonlace.net/m3/m3-bootstrap.AMD64.FREEBSD.92.tar.bz2 > > > > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > > 274: foreach root in [ m3cc, bin ] > > to: > > 274: foreach root in [ bin ] > > Before building the Modula-3 head. > > > > >> caused the same problem to occur in a different way. Even when the > >> newly rebuilt cm3cg executable was not copied to the bin directory > >> right away, its mere existence where it was built was causing it to > >> be found the next time cm3 started up, leading to the same failure. > >> > >> I don't remember how this was fixed, but either Jay or I did something > >> about it. Perhaps the ports bootstrap was made during a time when this > >> behavior was happening. > > > > Probably not as I am 99% sure the bootstrap was built from the release > > source. > > > > Thanks, > > John > > > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Mon Jun 1 09:11:22 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 1 Jun 2015 09:11:22 +0200 (CEST) Subject: [M3devel] duplicate subscription Message-ID: It seems that I am subscribed twice to this list. How can I find out the according e-mail addresses? From adacore at marino.st Mon Jun 1 09:19:01 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 09:19:01 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C04AD.5000509@marino.st> References: <556700B0.8060500@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> Message-ID: <556C0765.5050607@marino.st> On 6/1/2015 09:07, John Marino wrote: > On 6/1/2015 00:51, Jay wrote: >> Ps. Update pkginfo.txt and then try make-dist.py >> > > Okay, I applied this patch: > http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt > > and the same thing happened, only later: > http://leaf.dragonflybsd.org/~marino/m3e.log > Well, not the "same" thing. ${STAGE}/compiler_with_self/bin seems to be installed. I didn't change bootstrap/etc/modula3/cm3cfg.common though, it sounds like I shouldn't after all. Should I? John From jay.krell at cornell.edu Mon Jun 1 09:35:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:35:19 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C0765.5050607@marino.st> References: <556700B0.8060500@marino.st>, <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st> Message-ID: Given that you seem to be very quick and very patient, then yes, try that. I am trying to get back up to speed in all this, but you are way ahead of me. The actual fix in 2010 was more than Rodney described. It was removing a bunch of code. The theory is that we are still reaching around and finding the wrong cm3cg somewhere, instead of going directly to the matching one in the one place it belongs (ignoring cross build matters). It isn't entirely clear though, like where would it find the wrong one. I understand how it used to go bad . see: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common I hope to get catch up soon.. :( on being away.. - Jay > Date: Mon, 1 Jun 2015 09:19:01 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 09:07, John Marino wrote: > > On 6/1/2015 00:51, Jay wrote: > >> Ps. Update pkginfo.txt and then try make-dist.py > >> > > > > Okay, I applied this patch: > > http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt > > > > and the same thing happened, only later: > > http://leaf.dragonflybsd.org/~marino/m3e.log > > > > > Well, not the "same" thing. > ${STAGE}/compiler_with_self/bin seems to be installed. > > I didn't change bootstrap/etc/modula3/cm3cfg.common though, it sounds > like I shouldn't after all. Should I? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Jun 1 10:02:13 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 1 Jun 2015 10:02:13 +0200 Subject: [M3devel] duplicate subscription In-Reply-To: References: Message-ID: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> On Mon, 1 Jun 2015 09:11:22 +0200 (CEST) Henning Thielemann wrote: > > It seems that I am subscribed twice to this list. How can I find out the > according e-mail addresses? Mailing list information can be found at https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel (or /m3commit, /m3announce). Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From adacore at marino.st Mon Jun 1 10:04:40 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:04:40 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st> Message-ID: <556C1218.3000103@marino.st> On 6/1/2015 09:35, Jay K wrote: > Given that you seem to be very quick and very patient, then yes, try that. > I am trying to get back up to speed in all this, but you are way ahead > of me. After this morning, I'll have to put this away until the weekend I think. However, changing that file in addition almost worked: http://leaf.dragonflybsd.org/~marino/m3f.log Failed at the end of log: "/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib/src/../../quake/cvsup.quake", line 117: quake runtime error: undefined variable: IsDarwin I'm guessing a simple patch to cvsup.quake would be sufficient, although it would me the error is current. John From wagner at elegosoft.com Mon Jun 1 10:13:09 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 1 Jun 2015 10:13:09 +0200 Subject: [M3devel] cm3/m3quake: setting environment variables for exec In-Reply-To: <556B49DD.20909@elstel.org> References: <556B49DD.20909@elstel.org> Message-ID: <20150601101309.2ac490ee3305b476a29be2ec@elegosoft.com> On Sun, 31 May 2015 19:50:21 +0200 Elmar Stellnberger wrote: > How can I set environment variables for exec with cm3? > Using /usr/bin/env would be inherently non-portable. > With SRC M3 I remember this was the 4th parameter or so. > However cm3 exec seems to do something very different. I don't know about that. The complete quake specification can be found at http://www.opencm3.net/doc/help/cm3/quake.html including all built-in functions and procedures. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 1 10:15:35 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 08:15:35 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1218.3000103@marino.st> References: <556700B0.8060500@marino.st>, <20150529122732.1396eb079ff9b404bcb21e40@elego.de>, <5568421D.3050209@marino.st>, <20150529155833.8454664b88391d93708e8f85@elego.de>, <55687266.3090400@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, <556885FD.6010800@marino.st>, <20150529180223.d467621b68ebf665da685b12@elego.de>, <55689065.1040207@marino.st>, <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B213E.8030108@marino.st>, , <556B4BE9.1000902@marino.st>, , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , <556C1218.3000103@marino.st> Message-ID: This is very good progress. Thank you for your speed and patience. And the error is kind of minor and uninteresting. Yes, I understand, it is fatal. Everything should just work to completion. See IsDarwin here: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common cvsup is fiddling around how to use zlib. See here: https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile and surely cvsup isn't interesting any more? Who is using cvs? You can delete m3-tools/cvsup. or change the scripts to skip it or update cm3cfg.common to be more up to date. which brings me to...seems like maybe an incorrect mixing of bootstrap config and latest config? Which begs the question though to which parts of config are local-specific and shouldn't be updated/replaced, and which parts are bound to cm3 and should be replaced. Which really suggest a slight refactoring of the config files. They are nicely factored, but not quite along these lines. Arguably whatever is bound to cm3 should be written in Modula-3 and statically linked, not in these free floating text files. But quake is convenient, and fast enough seemingly... - Jay > Date: Mon, 1 Jun 2015 10:04:40 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 09:35, Jay K wrote: > > Given that you seem to be very quick and very patient, then yes, try that. > > I am trying to get back up to speed in all this, but you are way ahead > > of me. > > After this morning, I'll have to put this away until the weekend I think. > > However, changing that file in addition almost worked: > http://leaf.dragonflybsd.org/~marino/m3f.log > > Failed at the end of log: > "/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib/src/../../quake/cvsup.quake", > line 117: quake runtime error: undefined variable: IsDarwin > > > I'm guessing a simple patch to cvsup.quake would be sufficient, although > it would me the error is current. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 10:25:11 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:25:11 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <5568421D.3050209@marino.st>, <20150529155833.8454664b88391d93708e8f85@elego.de>, <55687266.3090400@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, <556885FD.6010800@marino.st>, <20150529180223.d467621b68ebf665da685b12@elego.de>, <55689065.1040207@marino.st>, <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B213E.8030108@marino.st>, , <556B4BE9.1000902@marino.st>, , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , <556C1218.3000103@marino.st> Message-ID: <556C16E7.505@marino.st> On 6/1/2015 10:15, Jay K wrote: > This is very good progress. > Thank you for your speed and patience. > > > And the error is kind of minor and uninteresting. Yes, I understand, it > is fatal. Everything should just work to completion. > > > See IsDarwin here: > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > cvsup is fiddling around how to use zlib. > See here: > https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile > > > and surely cvsup isn't interesting any more? Who is using cvs? NetBSD and OpenBSD for starters. > You can delete m3-tools/cvsup. > or change the scripts to skip it > or update cm3cfg.common to be more up to date. I'd like to keep it, it's the best source for cvsup. I thought make-dist.py was just going to bootstrap (e.g. minimal required). Is it actually building everything? I was intending to run do-cm3-min.sh buildship, do-cm3-std.sh buildship, do-cm3-caltech-parser.sh buildship after make-dist.py completed. John From dmuysers at hotmail.com Mon Jun 1 10:26:37 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Mon, 1 Jun 2015 10:26:37 +0200 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <20150601002944.GA10751@topoi.pooq.com> References: <20150601002944.GA10751@topoi.pooq.com> Message-ID: Yes, there is libiup. Interface is portable C. Remains to write an M3 interface file. And it is pure Unicode (UTF-8), as it should be everywhere -----Original Message----- From: Hendrik Boom Sent: Monday, June 01, 2015 2:29 AM To: m3devel Subject: [M3devel] User interfaces for Modula 3. What user interface libraries are available for Modula 3? I know there's Trestle. But is there something like GTK or QT or somethng I can use like cairo to draw really pretty text? Anything that supports lots of Unicode? I don't mind havein to do my own character placement based on font metrics of some sort; not now, bit later, I may have some really weird layout constraints. In case you're wodering about the application: I'm doing preliminary planning for something like a text editor with several windows, one with the raw text, and another with a continuous (as continuous as I have cpu cycles for) display of the results of rather complex calculations on that text (such as proof checking or described graphics). Modula 3 isn't the only candidate for a programming language, just the leading candidate for historical and bootstrapping reasons. The others at the moment are OCAML and some kind of statically typed Scheme. And of course, I may decide that theo whole projecct is infeasible. -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: wlEmoticon-smile[1].png Type: image/png Size: 1046 bytes Desc: not available URL: From adacore at marino.st Mon Jun 1 10:58:59 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:58:59 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> Message-ID: <556C1ED3.3000303@marino.st> On 6/1/2015 10:53, Jay K wrote: > make-dist makes everything. > It makes a "min" distribution and an "all" distribution. > Hopefully you can read it through it makes semi-reasonable sense. yes, I already figured this out. > This is all my doing -- which is to say, I quite like it, > but not necessarily anyone else. > min as I recall, is just the compiler: cm3, cm3cg, mklib, config, > m3core, libm3 > (mklib is only for NT386 target, but it is small and innocuous) > all is everything, including min > I don't like the confusion and purported complexity of breaking > things down into several medium sized packages, with dependencies. > I want more of a "one stop shop", granted, a big one, that > gives everyone stuff they won't use. > But I compromised and it makes to things. Right now, I'm assuming I can write an "install target" that copies from $STAGE/$DISTRIBUTION to where it needs to be installed in ports. If so, this approach is fine. Simpler even. > > cvs is pretty awful. > I'm still more productive in it than git, but I expect that will change. > But ok. cvsup can work. The error is for a trivial thing. I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and hopefully that will work. I don't know quake language. (I don't know why IsDarwin does't work because it's defined in the new cm3 distibution's cm3cfg.common.) Let's see how it goes. John From jay.krell at cornell.edu Mon Jun 1 10:53:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 08:53:00 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C16E7.505@marino.st> References: <556700B0.8060500@marino.st>, <5568421D.3050209@marino.st>, , <20150529155833.8454664b88391d93708e8f85@elego.de>, , <55687266.3090400@marino.st>, , <20150529162028.388911007a614006049beb42@elego.de>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> Message-ID: make-dist makes everything. It makes a "min" distribution and an "all" distribution. Hopefully you can read it through it makes semi-reasonable sense. This is all my doing -- which is to say, I quite like it, but not necessarily anyone else. min as I recall, is just the compiler: cm3, cm3cg, mklib, config, m3core, libm3 (mklib is only for NT386 target, but it is small and innocuous) all is everything, including min I don't like the confusion and purported complexity of breaking things down into several medium sized packages, with dependencies. I want more of a "one stop shop", granted, a big one, that gives everyone stuff they won't use. But I compromised and it makes to things. It makes roughly speaking .tar.gz files. Or .zips for some targets. Or compresed tar, depending on what compressor it can find, and bundles them into .deb files. But .deb files are little more than .tar.gz or .tar.bz2 or .tar.xz upgrade.sh also builds everything When I wrote it to upgrade.py I missed that significant aspect, and it only builds the compiler. Perhaps I should rename it to upgrade-min.py and then have upgrade-all.py. You can mimic upgrade.sh with ugprade.py && do-cm3-all.py realclean skipgcc && do-cm3-all.py buildship (skipgcc is just an optimization to avoid some duplicate work) I never use any of the .sh files. I found them to hard to understand and not portable enough. i.e. I'd rather install Python on Windows than Cygwin. I suspect Bourne shell is just not worth learning to program in. I had rewritten them in .cmd for Windows, but wanted one set of scripts. cvs is pretty awful. I'm still more productive in it than git, but I expect that will change. But ok. cvsup can work. The error is for a trivial thing. - Jay > Date: Mon, 1 Jun 2015 10:25:11 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 10:15, Jay K wrote: > > This is very good progress. > > Thank you for your speed and patience. > > > > > > And the error is kind of minor and uninteresting. Yes, I understand, it > > is fatal. Everything should just work to completion. > > > > > > See IsDarwin here: > > > > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > > > > cvsup is fiddling around how to use zlib. > > See here: > > https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile > > > > > > and surely cvsup isn't interesting any more? Who is using cvs? > > NetBSD and OpenBSD for starters. > > > > > You can delete m3-tools/cvsup. > > or change the scripts to skip it > > or update cm3cfg.common to be more up to date. > > I'd like to keep it, it's the best source for cvsup. > > I thought make-dist.py was just going to bootstrap (e.g. minimal > required). Is it actually building everything? > > > I was intending to run do-cm3-min.sh buildship, do-cm3-std.sh buildship, > do-cm3-caltech-parser.sh buildship after make-dist.py completed. > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 11:11:06 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:11:06 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1ED3.3000303@marino.st> References: <556700B0.8060500@marino.st>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> Message-ID: <556C21AA.70501@marino.st> On 6/1/2015 10:58, John Marino wrote: > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > hopefully that will work. I don't know quake language. quake doesn't know "False". If "0" doesn't work, I'll have to break down and come up with a patch to remove these last 3 lines. :) From jay.krell at cornell.edu Mon Jun 1 11:13:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 09:13:41 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C21AA.70501@marino.st> References: <556700B0.8060500@marino.st>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, ,,<556885FD.6010800@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, ,,<55689065.1040207@marino.st>, ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, ,, <556B213E.8030108@marino.st>,,, , ,, <556B4BE9.1000902@marino.st>,,, , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, ,,, ,,<556C1218.3000103@marino.st> , , <556C16E7.505@marino.st> , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st> Message-ID: try FALSE Here is some sample code: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common A patch shouldn't be needed but... hopefully soon I can try to duplicate what you are doing.. - Jay > Date: Mon, 1 Jun 2015 11:11:06 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 10:58, John Marino wrote: > > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > > hopefully that will work. I don't know quake language. > > quake doesn't know "False". If "0" doesn't work, I'll have to break > down and come up with a patch to remove these last 3 lines. :) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Mon Jun 1 11:24:56 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 1 Jun 2015 11:24:56 +0200 (CEST) Subject: [M3devel] duplicate subscription In-Reply-To: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> References: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> Message-ID: On Mon, 1 Jun 2015, Olaf Wagner wrote: > On Mon, 1 Jun 2015 09:11:22 +0200 (CEST) > Henning Thielemann wrote: > >> >> It seems that I am subscribed twice to this list. How can I find out the >> according e-mail addresses? > > Mailing list information can be found at > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel I know how to unsubscribe, but in order to do so I have to know the precise e-mail address I am subscribed with. I don't know them. :-( From adacore at marino.st Mon Jun 1 11:26:37 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:26:37 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , , <55689065.1040207@marino.st>, , , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , <556B213E.8030108@marino.st>, , , , , , <556B4BE9.1000902@marino.st>, , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , <556C1218.3000103@marino.st> , , <556C16E7.505@marino.st> , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st> Message-ID: <556C254D.7070306@marino.st> On 6/1/2015 11:13, Jay K wrote: > try FALSE > > Here is some sample code: > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > A patch shouldn't be needed but... hopefully soon I can try to duplicate > what you are doing.. Okay, I'll try "FALSE". That cm3cfg.common is pretty much what was installed at $STAGE/cm3-(all|min)-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/config/cm3cfg.common so I don't know why IsDarwin is not defined. === wait === okay, it failed again, same error (doesn't know IsDarwin), line 49 of m3-tools/cvsup/suplib/src/m3makefile So it seems we need to figure out why IsDarwin isn't known.... John From jay.krell at cornell.edu Mon Jun 1 11:29:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 09:29:54 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C254D.7070306@marino.st> References: <556700B0.8060500@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, , ,,<55689065.1040207@marino.st>, , ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , ,, <556B213E.8030108@marino.st>,,, , , , ,, <556B4BE9.1000902@marino.st>,,, , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , ,,, , , , <556C1218.3000103@marino.st> , ,,<556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> Message-ID: can you do like: export PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH cd /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib cm3 -commands -verbose cm3 -commands -verbose realclean cm3 -commands -verbose build - Jay > Date: Mon, 1 Jun 2015 11:26:37 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:13, Jay K wrote: > > try FALSE > > > > Here is some sample code: > > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > > A patch shouldn't be needed but... hopefully soon I can try to duplicate > > what you are doing.. > > Okay, I'll try "FALSE". > That cm3cfg.common is pretty much what was installed at > $STAGE/cm3-(all|min)-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/config/cm3cfg.common > so I don't know why IsDarwin is not defined. > > === wait === > > okay, it failed again, same error (doesn't know IsDarwin), line 49 of > m3-tools/cvsup/suplib/src/m3makefile > > So it seems we need to figure out why IsDarwin isn't known.... > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 11:44:02 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:44:02 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , <556B213E.8030108@marino.st>, , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , <556C1218.3000103@marino.st> , , , <556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> Message-ID: <556C2962.3020505@marino.st> On 6/1/2015 11:29, Jay K wrote: > can you do like: > export > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > cd > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > cm3 -commands -verbose > cm3 -commands -verbose realclean > cm3 -commands -verbose build It did not like "realclean" or "build": http://leaf.dragonflybsd.org/~marino/cvsupcmd.log John From jay.krell at cornell.edu Mon Jun 1 16:38:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 14:38:44 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C2962.3020505@marino.st> References: <556700B0.8060500@marino.st>, , ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , ,, <556B213E.8030108@marino.st>,,, , , , , , ,, <556B4BE9.1000902@marino.st>,,, , , , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , ,,, , , , , <556C1218.3000103@marino.st> , , ,,<556C16E7.505@marino.st> ,,, <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st> Message-ID: Sorry, multiple mistakes there by me. But I think I see the problem maybe. See it is always running mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 ? and never /mech/construction/mech/ptrees/default/lang/modula3/work/intermediate/compiler_with_{previous,self}/bin/{cm3,cm3cg} ? Do you have $CM3 set? Don't do that. It is supposed to be updating $PATH as it goes and finding cm3 and maybe cm3cg from there. But an environment variable CM3 will break it. https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py the function Setup That should print more, and maybe run and print the output of "where/which cm3" and "where/which cm3cg". and: https://github.com/modula3/cm3/blob/master/scripts/python/pylib.py: CM3 = ConvertPathForPython(getenv("CM3")) or "cm3" CM3 = SearchPath(CM3) I really just need to get and try your automation. You've made it available already? I'll have FreeBSD running locally "soon".. The right request by me would be to set PATH and CM3_INSTALL appropriately as make-dist.py would and run just building cvsup with -commands -verbose. Can you see how to do that? - Jay > Date: Mon, 1 Jun 2015 11:44:02 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:29, Jay K wrote: > > can you do like: > > export > > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > > cd > > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > > > cm3 -commands -verbose > > cm3 -commands -verbose realclean > > cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Jun 1 18:40:30 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 11:40:30 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C2962.3020505@marino.st> References: <556700B0.8060500@marino.st>, , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , <556B213E.8030108@marino.st>, , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , <556C1218.3000103@marino.st> , , , <556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> Message-ID: <556C8AFE.1050204@lcwb.coop> I am getting ready to write more on this, but a quick response: For the do-cm3-*.sh scripts, the options are realclean, build, buildship, etc. For cm3, the (supposedly) corresponding options are -realclean, -build, -buildship. This burns me every time I switch from one command form to the other. On 06/01/2015 04:44 AM, John Marino wrote: > On 6/1/2015 11:29, Jay K wrote: >> can you do like: >> export >> PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH >> cd >> /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib >> >> cm3 -commands -verbose >> cm3 -commands -verbose realclean >> cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 19:00:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 12:00:00 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1ED3.3000303@marino.st> References: <556700B0.8060500@marino.st>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> Message-ID: <556C8F90.9030907@lcwb.coop> On 06/01/2015 03:58 AM, John Marino wrote: > On 6/1/2015 10:53, Jay K wrote: >> make-dist makes everything. >> It makes a "min" distribution and an "all" distribution. >> Hopefully you can read it through it makes semi-reasonable sense. > > yes, I already figured this out. > >> This is all my doing -- which is to say, I quite like it, >> but not necessarily anyone else. >> min as I recall, is just the compiler: cm3, cm3cg, mklib, config, >> m3core, libm3 >> (mklib is only for NT386 target, but it is small and innocuous) >> all is everything, including min >> I don't like the confusion and purported complexity of breaking >> things down into several medium sized packages, with dependencies. >> I want more of a "one stop shop", granted, a big one, that >> gives everyone stuff they won't use. >> But I compromised and it makes to things. > > > Right now, I'm assuming I can write an "install target" that copies from > $STAGE/$DISTRIBUTION to where it needs to be installed in ports. If so, > this approach is fine. Simpler even. > > >> >> cvs is pretty awful. >> I'm still more productive in it than git, but I expect that will change. >> But ok. cvsup can work. The error is for a trivial thing. > > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > hopefully that will work. I don't know quake language. > Replace them with "", empty string. > (I don't know why IsDarwin does't work because it's defined in the new > cm3 distibution's cm3cfg.common.) > Yes, that surprises me too. suggests the new cm3cfg.common isn't getting interpreted. > Let's see how it goes. > John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 19:48:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 12:48:34 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , <556BD14D.5050507@lcwb.coop> Message-ID: <556C9AF2.7090704@lcwb.coop> On 06/01/2015 01:40 AM, Jay K wrote: > Yes, exactly, that's what I was trying to say, in my later/last mail. > > > Yes the compability is definitely two-way. > There is a one to one mapping back and forth. > cm3 goes with cm3cg; cm3cg goes with cm3. > Neither is compatible with older nor necessarily newer versions. > > > They would be statically linked together if not for the GPL. > > > When I first placed m3cc in pkginfo.txt years ago, I was only thinking > of build order and, like, "override". > > > m3cc imports nothing and nothing imports m3cc. > In the Modula-3 module/library/interface sense. > > > It can be built when you have nothing. > So I figured put it nearly first. > > > Only now today did I consider the shipping order. > It should be shipped adjacent to cm3, either right before or right after. > > > If you split build and ship, then it can still be built early, > or at almost any time. > Yes, this is the way I want to do building. First build everything you want to without putting any of it into use as build tools. Then put it all into use. One annoyance we currently have is that the do-cm3.*.sh scripts have, perhaps we should call them subcommands, like realclean, build, ship, buildship, etc. But the cm3 command treats their counterparts as command line options, -realclean, -build, -ship, -buildship, etc. I would like to consistify this. Removing the hyphen from cm3 would create ambiguities with file names, so it would mean adding the hyphen to the scripts. Much more serious is that similarly-named options do not have the same meaning in the scripts and in cm3. cm3 -build followed by cm3 -ship works as I expect, but do-cm3-*.sh build causes me nothing but trouble, and I learned long ago to avoid it. Instead, I always use do-cm3-*.sh buildship. This ships almost everything as it is built, but usually only cm3 and cm3cg executables matter, and today, they are specifically excepted from shipping, if you do not set INSTALL_CM3_IN_BIN="yes". I am writing from memory, but I think the problem with do-cm3-*.sh build is that build also uses overrides, which I barely understand, except that they, at best make things unshippable, and I think there was some other problem too. If we made the override thing a separate option to the scripts, so just build would neither ship nor override, I think that would satisfy my need, at least for the time being. And it would be consistent. Actually, I think we do want to use just-rebuilt packages when linking to them. Only executables and things they link in (possibly dynamically) need to be left alone until a later ship step. This usually only means cm3, cm3cg, and, if dynamically linked in, m3core. But there are others to think about, e.g., m3bundle, stubgen, netobjgen. > > But we have one ordering for build and ship, so let the shipping order decide. > > > I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. I have no objection to addressing the copy-over-an-executing-executable problem in a different way, and no objection to getting rid of the environment variable. But I do want it, for now at least, to be possible to get it to not ship either cm3 or cm3cg, even with the buildship option. If it just unconditionally doesn't ship them, that is fine. Before I added the variable check for the cm3cg package, it was unconditionally shipping it. > I still think it broke upgrade.py. > I can workaround. But I'd like to remove the environment variable check. > > > There are many order dependencies in build and ship. This is just one. > > > - Jay > > > > > > Date: Sun, 31 May 2015 22:28:13 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; adacore at marino.st > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > > to fail regardless of the order of building the two. > > > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > > it should work, because m3cc contains no Modula-3 code, and the new > > cm3 would not, when building it, run cm3cg. After that, there would > > be a consistent set. for further compiles. But any package containing > > M3 code built between cm3 and m3cc would fail. > > > > On 05/31/2015 05:15 PM, Jay wrote: > > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > > > - Jay > > > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > >> Should I put it back? From the last response I'm guessing that wasn't > > >> the problem. > > >> > > >> John > > >> > > >> > > >> On 5/31/2015 19:09, Jay K wrote: > > >>> Oh probably the problem is using the backend premature actually. > > >>> Darn, I can't help right now. > > >>> > > >>> - Jay > > >>> > > >>> > > >>> > > >>> ------------------------------------------------------------------------ > > >>> From: jay.krell at cornell.edu > > >>> To: adacore at marino.st; m3devel at elegosoft.com > > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > >>> (0x100), expected 0x110 > > >>> > > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > > >>> but my read was it didn't try that. > > >>> So.. with INSTALL_CM3_IN_BIN=yes? > > >>> I need to try this all, and "fix" it to set that. The whole thing in > > >>> m3cc/src/m3makefile is relatively > > >>> new vs. when I was actively using all this. :( > > >>> > > >>> - Jay > > >>> > > >>> > > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > > >>>> From: adacore at marino.st > > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > >>> (0x100), expected 0x110 > > >>>> > > >>>> On 5/31/2015 11:53, Jay K wrote: > > >>>>> John, have you tried make-dist.py? > > >>>>> i.e. > > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > > >>>>> > > >>>>> While it might not be exactly what you need, it should demonstrate the > > >>>>> elements. > > >>>>> > > >>>>> I will try to try this all soon, really, I hope so. > > >>>>> > > >>>>> It starts with a working compiler, does no in-place updates, build into > > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > > >>>>> It does "min" and "all". > > >>>>> > > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > > >>>>> > > >>>>> - Jay > > >>>> > > >>>> Hi Jay, > > >>>> I'm getting the same error as always using make-dist script: > > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > > >>>> > > >>>> Thanks, > > >>>> John > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 21:47:48 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 19:47:48 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C9AF2.7090704@lcwb.coop> References: <556700B0.8060500@marino.st>,<55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , , , <556BD14D.5050507@lcwb.coop>, , <556C9AF2.7090704@lcwb.coop> Message-ID: Sorry I'm rushing here..: Imho, build should build and ship should ship and buildship should do both. If that isn't the case, we should fix. Then, if you don't want to ship something, don't ship it. Not, ship it and set the environment variable. >implied: What is override? There are essentially two package pools, or sets of libraries, to chose from. They are roughly speaking the installed/shipped stuff: /usr/local/cm3/pkg/m3core/target (obviously it can vary) and the locally built but not yet shipped stuff, I call them: $HOME/dev2/cm3/m3-libs/m3core/target (obviously it can vary) I also call this "the object tree", i.e. the tree with a bunch of .obj/.o files. Or the "intermediates" tree, i.e. a bunch of outputs that are pruned out by "setup", and the outputs that are ultimately interesting The default is the first, override means to use the second. Then, the idea is that only stuff built against the install is safe to overwrite/add to the install. But I think is missing there is that, well, if I've built the entire system with overrides and I'm going to ship the entire system, then ship is safe. This isn't the best approach imho. Arguably better approaches are: always have a 2-level search first the object tree then the install tree This breaks people who switch around between incompatible changes in m3core and random application development, with the same source/object tree. However maybe that is just tough. The object tree doesn't have to be in the source tree. You should be able to say, like: symlinktree /usr/local/cm3 /working-on-feature1 export/set CM3_SOMETHING=/working-on-feature1 and case I change cm3 also export/set PATH=/working-on-feature1/bin:$PATH and then copy on write as you ship and then to switch to some other work: symlinktree /usr/local/cm3 /working-on-feature2 export/set CM3_SOMETHING=/working-on-feature2 and case I change cm3 also export/set PATH=/working-on-feature2/bin:$PATH and then copy on write as you ship, different stuff That is a working model that I advocate but don't yet follow. Though make-dist kind of does. make-dist copies the compiler to the new location and then does buildship. In this model, you don't have overrides and you always build/ship. And if you mess up, you can start over easily. But I haven't actually established workflow here so I'm just talking too much. : A real analog in other people's workflows is: mkdir /obj/working-on-feature1 cd /obj/working-on-feature1 /src/configure ... make mkdir /obj/working-on-feature2 cd /obj/working-on-feature2 /src/configure ...potentially different .. make .sh build should not ship. Agreed. Currently if you are going to ship, you can't override. make-dist.py uses a destdir-like approach. There is some contention perhaps of scenarios of "I'm just building a little, small safe changes" and "I'm rebuilding everything, and possibly messing up". I really want the environment variable checks to go away. - Jay > Date: Mon, 1 Jun 2015 12:48:34 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 06/01/2015 01:40 AM, Jay K wrote: > > Yes, exactly, that's what I was trying to say, in my later/last mail. > > > > > > Yes the compability is definitely two-way. > > There is a one to one mapping back and forth. > > cm3 goes with cm3cg; cm3cg goes with cm3. > > Neither is compatible with older nor necessarily newer versions. > > > > > > They would be statically linked together if not for the GPL. > > > > > > When I first placed m3cc in pkginfo.txt years ago, I was only thinking > > of build order and, like, "override". > > > > > > m3cc imports nothing and nothing imports m3cc. > > In the Modula-3 module/library/interface sense. > > > > > > It can be built when you have nothing. > > So I figured put it nearly first. > > > > > > Only now today did I consider the shipping order. > > It should be shipped adjacent to cm3, either right before or right after. > > > > > > If you split build and ship, then it can still be built early, > > or at almost any time. > > > > Yes, this is the way I want to do building. First build everything you want to > without putting any of it into use as build tools. Then put it all into use. > > One annoyance we currently have is that the do-cm3.*.sh scripts have, perhaps > we should call them subcommands, like realclean, build, ship, buildship, etc. > But the cm3 command treats their counterparts as command line options, > -realclean, -build, -ship, -buildship, etc. I would like to consistify > this. Removing the hyphen from cm3 would create ambiguities with file names, > so it would mean adding the hyphen to the scripts. > > Much more serious is that similarly-named options do not have the same meaning > in the scripts and in cm3. cm3 -build followed by cm3 -ship works as I expect, > but do-cm3-*.sh build causes me nothing but trouble, and I learned long ago to > avoid it. Instead, I always use do-cm3-*.sh buildship. This ships almost everything > as it is built, but usually only cm3 and cm3cg executables matter, and today, they are > specifically excepted from shipping, if you do not set INSTALL_CM3_IN_BIN="yes". > > I am writing from memory, but I think the problem with do-cm3-*.sh build is that > build also uses overrides, which I barely understand, except that they, at best > make things unshippable, and I think there was some other problem too. If we > made the override thing a separate option to the scripts, so just build would > neither ship nor override, I think that would satisfy my need, at least for the > time being. And it would be consistent. > > Actually, I think we do want to use just-rebuilt packages when linking to them. > Only executables and things they link in (possibly dynamically) need to be > left alone until a later ship step. This usually only means cm3, cm3cg, > and, if dynamically linked in, m3core. But there are others to think about, > e.g., m3bundle, stubgen, netobjgen. > > > > > But we have one ordering for build and ship, so let the shipping order decide. > > > > > > I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. > > I have no objection to addressing the copy-over-an-executing-executable problem > in a different way, and no objection to getting rid of the environment variable. > But I do want it, for now at least, to be possible to get it to not ship either > cm3 or cm3cg, even with the buildship option. If it just unconditionally doesn't > ship them, that is fine. Before I added the variable check for the cm3cg package, > it was unconditionally shipping it. > > > I still think it broke upgrade.py. > > I can workaround. But I'd like to remove the environment variable check. > > > > > > There are many order dependencies in build and ship. This is just one. > > > > > > - Jay > > > > > > > > > > > Date: Sun, 31 May 2015 22:28:13 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; adacore at marino.st > > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > > > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > > > to fail regardless of the order of building the two. > > > > > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > > > it should work, because m3cc contains no Modula-3 code, and the new > > > cm3 would not, when building it, run cm3cg. After that, there would > > > be a consistent set. for further compiles. But any package containing > > > M3 code built between cm3 and m3cc would fail. > > > > > > On 05/31/2015 05:15 PM, Jay wrote: > > > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > > > > > - Jay > > > > > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > > > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > > >> Should I put it back? From the last response I'm guessing that wasn't > > > >> the problem. > > > >> > > > >> John > > > >> > > > >> > > > >> On 5/31/2015 19:09, Jay K wrote: > > > >>> Oh probably the problem is using the backend premature actually. > > > >>> Darn, I can't help right now. > > > >>> > > > >>> - Jay > > > >>> > > > >>> > > > >>> > > > >>> ------------------------------------------------------------------------ > > > >>> From: jay.krell at cornell.edu > > > >>> To: adacore at marino.st; m3devel at elegosoft.com > > > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > > > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > > >>> (0x100), expected 0x110 > > > >>> > > > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > > > >>> but my read was it didn't try that. > > > >>> So.. with INSTALL_CM3_IN_BIN=yes? > > > >>> I need to try this all, and "fix" it to set that. The whole thing in > > > >>> m3cc/src/m3makefile is relatively > > > >>> new vs. when I was actively using all this. :( > > > >>> > > > >>> - Jay > > > >>> > > > >>> > > > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > > > >>>> From: adacore at marino.st > > > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > > > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > > >>> (0x100), expected 0x110 > > > >>>> > > > >>>> On 5/31/2015 11:53, Jay K wrote: > > > >>>>> John, have you tried make-dist.py? > > > >>>>> i.e. > > > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > > > >>>>> > > > >>>>> While it might not be exactly what you need, it should demonstrate the > > > >>>>> elements. > > > >>>>> > > > >>>>> I will try to try this all soon, really, I hope so. > > > >>>>> > > > >>>>> It starts with a working compiler, does no in-place updates, build into > > > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > > > >>>>> It does "min" and "all". > > > >>>>> > > > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > > > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > > > >>>>> > > > >>>>> - Jay > > > >>>> > > > >>>> Hi Jay, > > > >>>> I'm getting the same error as always using make-dist script: > > > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > > > >>>> > > > >>>> Thanks, > > > >>>> John > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Jun 2 00:16:57 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 1 Jun 2015 18:16:57 -0400 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> Message-ID: <20150601221657.GA4281@topoi.pooq.com> On Mon, Jun 01, 2015 at 07:47:48PM +0000, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. Make sense to me. -- hendrik From hendrik at topoi.pooq.com Tue Jun 2 00:23:19 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 1 Jun 2015 18:23:19 -0400 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> Message-ID: <20150601222319.GA4410@topoi.pooq.com> On Mon, Jun 01, 2015 at 07:47:48PM +0000, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. This would mean there would have to be some kind of loading dok for things that have been built but not yet shipped. It probably exists already. -- hendrik From rodney_bates at lcwb.coop Tue Jun 2 02:54:45 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 19:54:45 -0500 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, , <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , , , <556BD14D.5050507@lcwb.coop>, , <556C9AF2.7090704@lcwb.coop> Message-ID: <556CFED5.80002@lcwb.coop> Here is a short-term proposal (i.e., without major reorganization) for the do-cm3*.sh scripts: 1) 'build' only builds, as we seem to agree it should. 2) a new option 'override' (and only 'override') causes an override build 3) a new option 'partialship' ships, as each package is done, things that will be needed to compile another package that does a quake import on the just-built package (I think this means static library, if any, and .M3WEB), but does not ship things that will be used to execute the just-built package (I think this means executable or dynamic library). I'm not sure right off hand which ship group things like interface source files, etc. belong in. Plus, just as conveniences: 4) buildship means the same as build and ship (I think this is already the case.) 5) buildpartialship means the same as build and partialship 6) All these options at least allow a leading hyphen, so they are like the cm3 command. I guess cm3 should also have a -shippartial On 06/01/2015 02:47 PM, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 2 09:11:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 2 Jun 2015 07:11:44 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, ,,,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , ,,,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , , , , <556B213E.8030108@marino.st>, , , , ,,, , , , , , , <556B4BE9.1000902@marino.st>, , , , ,,, , ,,,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , ,,,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , ,,,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , ,,,,, , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: John, this and your other problems should all be fixed now. I was able to reproduce them mostly.I didn't actually reproduce the cvsup problem, but I see in the log you did set CM3, and I did test with that, and hit the cm3cg version mismatch, and the problem is pretty clear -- my mistake. See: https://github.com/modula3/cm3/commit/a149d4b5e715d89b076c92e0b41d74f9b5ff56e4 https://github.com/modula3/cm3/commit/769784562d0e45b968e515c1412ed9247ec050ef https://github.com/modula3/cm3/commit/a0e44b08e687a8bba677c299ffa459993525944c Thank you for your patience and diligence. I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && make-dist.py or skip right to make-dist.py.There was an additional problem skipping right to make-dist.py that I fixed, at least for Darwin. I don't use any of the .sh files -- I don't want to read/maintain/learn that programming language as I suspect it isn't particularly good. - Jay From: jay.krell at cornell.edu To: adacore at marino.st CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 Date: Mon, 1 Jun 2015 14:38:44 +0000 Sorry, multiple mistakes there by me. But I think I see the problem maybe. See it is always running mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 ? and never /mech/construction/mech/ptrees/default/lang/modula3/work/intermediate/compiler_with_{previous,self}/bin/{cm3,cm3cg} ? Do you have $CM3 set? Don't do that. It is supposed to be updating $PATH as it goes and finding cm3 and maybe cm3cg from there. But an environment variable CM3 will break it. https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py the function Setup That should print more, and maybe run and print the output of "where/which cm3" and "where/which cm3cg". and: https://github.com/modula3/cm3/blob/master/scripts/python/pylib.py: CM3 = ConvertPathForPython(getenv("CM3")) or "cm3" CM3 = SearchPath(CM3) I really just need to get and try your automation. You've made it available already? I'll have FreeBSD running locally "soon".. The right request by me would be to set PATH and CM3_INSTALL appropriately as make-dist.py would and run just building cvsup with -commands -verbose. Can you see how to do that? - Jay > Date: Mon, 1 Jun 2015 11:44:02 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:29, Jay K wrote: > > can you do like: > > export > > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > > cd > > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > > > cm3 -commands -verbose > > cm3 -commands -verbose realclean > > cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Tue Jun 2 09:19:33 2015 From: adacore at marino.st (John Marino) Date: Tue, 02 Jun 2015 09:19:33 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>, , , , , , , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: <556D5905.4010304@marino.st> On 6/2/2015 09:11, Jay K wrote: > John, this and your other problems should all be fixed now. > > I was able to reproduce them mostly. > I didn't actually reproduce the cvsup problem, but I see in the log you > did set CM3, and I did test with that, and hit the cm3cg version > mismatch, and the problem is pretty clear -- my mistake. Thanks Jay! Don't worry about the mistake. I'm just happy that my use case was able to flush out some issues and get some interesting discussion going. Sure it's been hard-going from my side but the final result is an improvement for everyone. I'll see what I can test at night in a VM but I don't get home back to my "real" machine until late Friday night (CEDT). But after looking over the commits this seems promising. John From wagner at elegosoft.com Tue Jun 2 11:36:11 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 2 Jun 2015 11:36:11 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556CFED5.80002@lcwb.coop> References: <556700B0.8060500@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> Message-ID: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> On Mon, 01 Jun 2015 19:54:45 -0500 "Rodney M. Bates" wrote: > Here is a short-term proposal (i.e., without major reorganization) > for the do-cm3*.sh scripts: > > 1) 'build' only builds, as we seem to agree it should. > 2) a new option 'override' (and only 'override') causes an override build > 3) a new option 'partialship' ships, as each package is done, things that > will be needed to compile another package that does a quake import on the > just-built package (I think this means static library, if any, and .M3WEB), > but does not ship things that will be used to execute the just-built package > (I think this means executable or dynamic library). I'm not sure right off > hand which ship group things like interface source files, etc. belong in. I don't know how you will implement this and how it should fit into the M3 package concept, unless you ship to a completely different location, i.e. another package pool (but cm3 currently just supports one). In my opinion the package system relies on the invariant that a shipped (installed) package is completely consistent, i.e. the source code interfaces correspond to the intermediate code interfaces and to the binary equivalent of them (static and dynamic libraries). If I understand you correctly, you want to give up this consistency in favour of being able to do an easier bootstrap of the CM3 system. But bootstrapping a complex system is never easy and usually requires some special or tricky steps. Ordinary use of the system, i.e. all other applications, that just build on the standard tools, don't need this kind of steps. If I could design (and implement) a better cm3 builder, I'd have one with multiple package pools (shipping destinations, locations of installed packages) and an integrated builder that knows all about the package dependencies (so that we haven't to do that in scripts). The two points are the only shortcomings in the cm3 build system in my opinion. The prjm tools of the ComPact sources was a step in this direction: it can handle multiple pools of packages and their dependencies, but is language independent, therefore too complex, and never got integrated into the compiler builder. As for the shell scripts, I'm surprised that they have survived so long; they just got created on the fly to support the building and publication of the CM3 system as open source when we got the sources from Critical Mass. They were never intended as a general user interface, but only as tools for the CM3 maintainers, and later got used in the Hudson CI. BTW, if I find some time and access to suitable machines again, I'd really like to set up a new CI system on Jenkins interfacing with the Github infrastruture. But I won't make any promises here. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From lists at darko.org Tue Jun 2 14:05:48 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 2 Jun 2015 05:05:48 -0700 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> References: <556700B0.8060500@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> Message-ID: Why can't we design and build a cm3 builder that supports these requirements? Relying on scripts seems entirely broken and an anathema to the portability of the CM3 system. There seems to be enough experience to specify and design such a system and building it doesn't sound like it would be very difficult. I'm sure there would be enough volunteers to implement it since people's productivity is obviously being affected. Wouldn't we want to fix the problem for once and for all instead of endlessly hacking scripts? Having a robust, reliable and portable end-to-end build system would also make it easier for people wanting to port and enhance the compiler, which would be a boost for the project. On Tue, Jun 2, 2015 at 2:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > > > Here is a short-term proposal (i.e., without major reorganization) > > for the do-cm3*.sh scripts: > > > > 1) 'build' only builds, as we seem to agree it should. > > 2) a new option 'override' (and only 'override') causes an override build > > 3) a new option 'partialship' ships, as each package is done, things that > > will be needed to compile another package that does a quake import > on the > > just-built package (I think this means static library, if any, and > .M3WEB), > > but does not ship things that will be used to execute the just-built > package > > (I think this means executable or dynamic library). I'm not sure > right off > > hand which ship group things like interface source files, etc. > belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other > applications, > that just build on the standard tools, don't need this kind of steps. > > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 2 18:18:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 02 Jun 2015 11:18:03 -0500 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> Message-ID: <556DD73B.9050601@lcwb.coop> On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org From dmuysers at hotmail.com Tue Jun 2 18:39:35 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 2 Jun 2015 18:39:35 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556DD73B.9050601@lcwb.coop> References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop><20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: golang is an example of good package management. It`s interaction of the builder with github repositories is an interesting idea. -----Original Message----- From: Rodney M. Bates Sent: Tuesday, June 02, 2015 6:18 PM To: Olaf Wagner Cc: m3devel Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 2 19:48:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 2 Jun 2015 17:48:13 +0000 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop><20150602113611.66de95acf8cdb242c6672605@elegosoft.com>, <556DD73B.9050601@lcwb.coop>, Message-ID: > Yes, absolutely. We need a greatly reworked build system for multiple packages. > For building within a package, cm3's existing build is really quite sophisticated. Exactly my thoughts for years. quake is nice and declarative for each package, and currently does nothing for multiple packages. How far would we get with the following: include_dir to stitch together packages, the same as it stitches directories within packages. "Flush" the state at each Library() or Program() invocation? Library and Program have to be last currently? So that would work? Or they can be anywhere? Furthermore, target directory either be: 1) ./target -- might be what it'd do now, but is wrong choice 2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. cd anywhere, that is still it 3) under some new new root, like go up from cwd until you find special marker file; this would be building outside source tree entirely 4) same as currently, wherever is Library()/Program(), go ../ If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"? Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir. it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib. It looks like:dirs file: DIRS=a \ b \ c sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake I use this system a lot and quite like it.Though it is obscure. The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad. I think scons works this way too, the language being Python. Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program. Make sense? Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.) - Jay From: dmuysers at hotmail.com To: rodney.m.bates at acm.org; wagner at elegosoft.com Date: Tue, 2 Jun 2015 18:39:35 +0200 CC: m3devel at elegosoft.com Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 golang is an example of good package management. It`s interaction of the builder with github repositories is an interesting idea. -----Original Message----- From: Rodney M. Bates Sent: Tuesday, June 02, 2015 6:18 PM To: Olaf Wagner Cc: m3devel Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Jun 2 20:59:51 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 2 Jun 2015 14:59:51 -0400 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: <20150602185951.GA6137@topoi.pooq.com> On Tue, Jun 02, 2015 at 05:48:13PM +0000, Jay K wrote: > > Yes, absolutely. We need a greatly reworked build system for multiple packages. > > For building within a package, cm3's existing build is really quite sophisticated. > > Exactly my thoughts for years. quake is nice and declarative for each package, and currently > does nothing for multiple packages. > > > How far would we get with the following: > > > include_dir to stitch together packages, the same as it stitches > directories within packages. > > "Flush" the state at each Library() or Program() invocation? > > Library and Program have to be last currently? So that would work? > Or they can be anywhere? > > > Furthermore, target directory either be: > 1) ./target -- might be what it'd do now, but is wrong choice Probably the wrong place. ./cm3target might be slightly better. Or make it not hidden. Or have a quake conmend to set it. > 2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. > cd anywhere, that is still it > 3) under some new new root, like go up from cwd until you find > special marker file; this would be building outside source tree entirely Outside the source tree entirely would be good. I'd really like to be able to compile a source tree that's on a read-only file system. Or on a shadow file system. > 4) same as currently, wherever is Library()/Program(), go ../ > > If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"? > > Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir. > > it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib. > > It looks like:dirs file: DIRS=a \ b \ c > sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake > I use this system a lot and quite like it.Though it is obscure. > The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad. > > I think scons works this way too, the language being Python. Writing the cm3 stanzas to tell scons how to use cm3 sould be useful, too. > > Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program. > > Make sense? > Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.) > > - Jay > > > > > From: dmuysers at hotmail.com > To: rodney.m.bates at acm.org; wagner at elegosoft.com > Date: Tue, 2 Jun 2015 18:39:35 +0200 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > > > golang is an example of good > package management. > It`s interaction of the builder with github repositories is an interesting > idea. > > -----Original Message----- > From: Rodney M. Bates > Sent: Tuesday, June 02, 2015 6:18 PM > To: Olaf Wagner > Cc: m3devel > Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG > version stamp (0x100), expected 0x110 > > > > On 06/02/2015 04:36 AM, Olaf Wagner wrote: > > On Mon, 01 Jun 2015 19:54:45 -0500 > > "Rodney M. Bates" wrote: > > > >> Here is a short-term proposal (i.e., without major > reorganization) > >> for the do-cm3*.sh scripts: > >> > >> 1) 'build' only builds, as we seem to agree it should. > >> 2) a new option 'override' (and only 'override') causes an > override build > >> 3) a new option 'partialship' ships, as each package is done, > things that > >> will be needed to compile another > package that does a quake import on the > >> just-built package (I think this > means static library, if any, and .M3WEB), > >> but does not ship things that will > be used to execute the just-built package > >> (I think this means executable or > dynamic library). I'm not sure right off > >> hand which ship group things like > interface source files, etc. belong in. > > > > I don't know how you will implement this and how it should fit into > the > > M3 package concept, unless you ship to a completely different > location, > > i.e. another package pool (but cm3 currently just supports one). > > > > In my opinion the package system relies on the invariant that a > shipped > > (installed) package is completely consistent, i.e. the source > code > > interfaces correspond to the intermediate code interfaces and to > the > > binary equivalent of them (static and dynamic libraries). If I > understand > > you correctly, you want to give up this consistency in favour of > being > > able to do an easier bootstrap of the CM3 system. > > Yes, but we already must have a temporarily inconsistent state of > installation > today. This is a somewhat different kind of inconsistency, but > inconsistent, > nonetheless. > > I guess -partialship could do an additional final pass over the package > list, > doing a full ship. The inconsistent state would only last during a > single > user-requested step, which is about as short as it can get. My intent > was > that one would immediately do a full ship after a partialship, although > as > a separate command, thus restoring consistency. > > > > > > But bootstrapping a complex system is never easy and usually requires > some > > special or tricky steps. Ordinary use of the system, i.e. all other > applications, > > that just build on the standard tools, don't need this kind of > steps. > > > > And my proposal is a simple, short-term way to provide somewhat more > flexible > options for just such special or tricky steps. It will work the same > way as now, > if you don't use the new options, and would allow Jay to get rid of the > hated > INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special > steps. > > > If I could design (and implement) a better cm3 builder, I'd have one > with > > multiple package pools (shipping destinations, locations of > installed > > packages) and an integrated builder that knows all about the > package > > dependencies (so that we haven't to do that in scripts). The two > points > > are the only shortcomings in the cm3 build system in my opinion. > > The prjm tools of the ComPact sources was a step in this > direction: > > it can handle multiple pools of packages and their dependencies, > but > > is language independent, therefore too complex, and never got > integrated > > into the compiler builder. > > > > Yes, absolutely. We need a greatly reworked build system for multiple > packages. > For building within a package, cm3's existing build is really quite > sophisticated. > I don't know of any other that detects the need for recompilation > on declaration-granularity. But it does very little with > inter-package > problems -- just detection of link-time inconsistencies, with > hopelessly > uninformative error messages. (I once set out to improve them, but > it > required more info in the .mx and .m3x files, with more incompatible > changes, and I only got one message improved a little bit, before I > gave > up for the time.) > > I agree, reworking it all is very desirable. It will take a lot of > rework. > I think we all agree that some kind of multi-layer, multi-destination > scheme is needed. This will probably entail disentangling the > source > and build directories from being interspersed in each package, or > something > equally perturbing to the existing system. It isn't going to be a > little > patch. I think my proposal will be small to implement and have no > impact > on anything existing, if you don't use the new options. > > > > As for the shell scripts, I'm surprised that they have survived so > long; > > they just got created on the fly to support the building and > publication > > of the CM3 system as open source when we got the sources from > Critical > > Mass. They were never intended as a general user interface, but > only > > as tools for the CM3 maintainers, and later got used in the Hudson > CI. > > > > BTW, if I find some time and access to suitable machines again, > I'd > > really like to set up a new CI system on Jenkins interfacing with > the > > Github infrastruture. But I won't make any promises here. > > > > Yes, this would be very good. Priorities, priorities! :-(. > > > Olaf > > > > -- > Rodney Bates > rodney.m.bates at acm.org From wagner at elegosoft.com Tue Jun 2 21:19:52 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 2 Jun 2015 21:19:52 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556DD73B.9050601@lcwb.coop> References: <556700B0.8060500@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: <20150602211952.17478aefac4b66e13de704a9@elegosoft.com> On Tue, 02 Jun 2015 11:18:03 -0500 "Rodney M. Bates" wrote: > > > On 06/02/2015 04:36 AM, Olaf Wagner wrote: > > On Mon, 01 Jun 2015 19:54:45 -0500 > > "Rodney M. Bates" wrote: > > > >> Here is a short-term proposal (i.e., without major reorganization) > >> for the do-cm3*.sh scripts: > >> > >> 1) 'build' only builds, as we seem to agree it should. > >> 2) a new option 'override' (and only 'override') causes an override build > >> 3) a new option 'partialship' ships, as each package is done, things that > >> will be needed to compile another package that does a quake import on the > >> just-built package (I think this means static library, if any, and .M3WEB), > >> but does not ship things that will be used to execute the just-built package > >> (I think this means executable or dynamic library). I'm not sure right off > >> hand which ship group things like interface source files, etc. belong in. > > > > I don't know how you will implement this and how it should fit into the > > M3 package concept, unless you ship to a completely different location, > > i.e. another package pool (but cm3 currently just supports one). > > > > In my opinion the package system relies on the invariant that a shipped > > (installed) package is completely consistent, i.e. the source code > > interfaces correspond to the intermediate code interfaces and to the > > binary equivalent of them (static and dynamic libraries). If I understand > > you correctly, you want to give up this consistency in favour of being > > able to do an easier bootstrap of the CM3 system. > > Yes, but we already must have a temporarily inconsistent state of installation > today. This is a somewhat different kind of inconsistency, but inconsistent, > nonetheless. > > I guess -partialship could do an additional final pass over the package list, > doing a full ship. The inconsistent state would only last during a single > user-requested step, which is about as short as it can get. My intent was > that one would immediately do a full ship after a partialship, although as > a separate command, thus restoring consistency. OK, I think I have a better idea of what you want to do now. > > But bootstrapping a complex system is never easy and usually requires some > > special or tricky steps. Ordinary use of the system, i.e. all other applications, > > that just build on the standard tools, don't need this kind of steps. > > > > And my proposal is a simple, short-term way to provide somewhat more flexible > options for just such special or tricky steps. It will work the same way as now, > if you don't use the new options, and would allow Jay to get rid of the hated > INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > > > If I could design (and implement) a better cm3 builder, I'd have one with > > multiple package pools (shipping destinations, locations of installed > > packages) and an integrated builder that knows all about the package > > dependencies (so that we haven't to do that in scripts). The two points > > are the only shortcomings in the cm3 build system in my opinion. > > The prjm tools of the ComPact sources was a step in this direction: > > it can handle multiple pools of packages and their dependencies, but > > is language independent, therefore too complex, and never got integrated > > into the compiler builder. > > > > Yes, absolutely. We need a greatly reworked build system for multiple packages. > For building within a package, cm3's existing build is really quite sophisticated. > I don't know of any other that detects the need for recompilation > on declaration-granularity. But it does very little with inter-package > problems -- just detection of link-time inconsistencies, with hopelessly > uninformative error messages. (I once set out to improve them, but it > required more info in the .mx and .m3x files, with more incompatible > changes, and I only got one message improved a little bit, before I gave > up for the time.) > > I agree, reworking it all is very desirable. It will take a lot of rework. > I think we all agree that some kind of multi-layer, multi-destination > scheme is needed. This will probably entail disentangling the source > and build directories from being interspersed in each package, or something > equally perturbing to the existing system. It isn't going to be a little > patch. I think my proposal will be small to implement and have no impact > on anything existing, if you don't use the new options. I'm not against your proposal as a first or intermediate step. We have to remain pragmatic and improve things incrementally. If people really use the old shell scripts, I'm not against adapting them. Jay may do all the same things in Python even faster, but I always found that was an additional unnecessary dependency. On the other hand, Python can probably be installed in a couple of minutes on any system these days. I also agree that we would all like a multi-level fully integrated build system for sets of M3 packages, but that will be a lot of work and I won't be able to contribute much to it. Those who do the work will decide about the design and its capabilities. > > As for the shell scripts, I'm surprised that they have survived so long; > > they just got created on the fly to support the building and publication > > of the CM3 system as open source when we got the sources from Critical > > Mass. They were never intended as a general user interface, but only > > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > > > BTW, if I find some time and access to suitable machines again, I'd > > really like to set up a new CI system on Jenkins interfacing with the > > Github infrastruture. But I won't make any promises here. > > Yes, this would be very good. Priorities, priorities! :-(. It's not as improbable as me writing a new integrated build system though, as I've set up several complex CI systems for other projects. We'll se if I find the fime... Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Jun 3 03:58:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 01:58:10 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem Message-ID: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 04:51:45 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 02:51:45 +0000 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) Message-ID: Ideally: 1 32bit host could target 64bit target 2 32bit texts could be as large as the address space/OS allows. 3 ditto 64bit texts -- larger than 4GB 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. Currently only #4 is true. TEXTs for any target can only be about 500MB. To fix 2 and 3 requires m3front to use TInt more. This fixes #1 and likely breaks #4. Cutting the limit just a little. Can we special case somehow? Is there some construct we can use that has no stated limit, like 0..cnt - 1? Should/can we introduce one? jbook2:src jay$ git diff "../src/text/TextLiteral.i3"diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..4d16a44 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; CONST (* DIV BITSIZE should not be here! *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ otherwise we have: new source -> compiling TextLiteral.i3"../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large1 error encountered Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 08:17:49 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 08:17:49 +0200 Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>, , , , , , , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: <556E9C0D.5000403@marino.st> On 6/2/2015 09:11, Jay K wrote: > John, this and your other problems should all be fixed now. > > I was able to reproduce them mostly. > I didn't actually reproduce the cvsup problem, but I see in the log you > did set CM3, and I did test with that, and hit the cm3cg version > mismatch, and the problem is pretty clear -- my mistake. > > I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && > make-dist.py or skip right to make-dist.py. > There was an additional problem skipping right to make-dist.py that I > fixed, at least for Darwin. Hi Jay, I set up the VM last night and it completed the make-dist.py command. I have some questions / comments. comment 1: Using bin for config isn't a standard unix location. e.g. rather than /bin/config/AMD64_FREEBSD, it should be located at /etc/modula3/config/AMD64_FREEBSD (can it be configurable?) comment 2: Many of the binaries are might actually be example programs, e.g. cube, calculator, fisheye. Rather than /bin where they risk clashing with other programs, it might be advisable to stick them a /share/examples/modula3 (configurable?). comment 3: /www is not standard either. We would be this in /share/doc/modula3. Can it be configurable? Comments 1--3 might be related because www might have relative paths that need changing comment 4: I'd like a configurable option not to tar/gzip the products. In my case, I'm going to stage them manually so this is an unnecessary step. comment 5: the "min" distribution seems suitable to be packaged as a bootstrap question 6: In most use cases, other than intentionally recreating the bootstrap, I'm not going to need the "min" distribution. Is there any impact to building "all" only? In other words, does something else use "min" distribution? Thanks for the update! John From jay.krell at cornell.edu Wed Jun 3 09:18:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 07:18:41 +0000 Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) In-Reply-To: <556E9C0D.5000403@marino.st> References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>,,, , , , , , , , , ,,, , , <556B4BE9.1000902@marino.st>, , , , , ,,, , , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , ,,<556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, ,,, , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st> Message-ID: "Yes". 1) "config"'s user configurability is over-implied imho. It shouldn't all be considered config. It needs refactoring into bin and config. Possibly rewriting the stuff that has stood the test of a few year's time into native Modula-3. It used to be worse. Each file used to be standalone. I refactored -- all the "common" files. I think it is partly called config because if you look at the gcc source tree, it also has a config directory. https://github.com/modula3/cm3/tree/master/m3-sys/m3cc/gcc/gcc/config It should be called "target" perhaps. Heck, we could have bin//cm3cg, bin//cm3.cfg or bin/config.quake or such. Consider all the Perl and Python and sh you may or may not have in /usr/bin. Is it configurable? User editable? Maybe/kinda. Consider that /etc is often sh snippets. The line between code and data is somewhat blurry, and I am somewhat lazy. We could split the files into "less editable" and "more editable". The bulk of the text in the files is hard to understand and hard to edit. And then there is the question of updating. And when I make edits to "the defaults", which do I assume are carried with cm3, vs. which do I assume an update will leave alone? Basically, I'm lazy, and haven't perfectly evaluated or factored everything. It used to be worse. However, there is/was cminstall, which the user is/was meant to run which would edit small parts. So maybe those very parts should be broken out into separate files? There another problem though. Which parts of "config"/cminstall need to be rerun when something else changes? You know, imagine I install cm3 and then I install X Windows. Installing X Windows implies a need to reconfigure. The full needs of a configuration system are related to a packaging system. Debian I believe handles this all well, after years of development focusing on these sorts of things. And then we'd have to handle all the other packaging systems. And nobody has been interested/motivated enough so far to really integrate so well into all the various packaging systems. But also, we target not just Unix. Yet we have almost the same directory structure on Unix and Windows. The difference is that on Windows we put the .dlls in bin. Should we use $HOME/etc or $HOME/config? And have /usr/local/cm3/bin probe around? I think I did put in some probing. To what extend do users just have private installs $HOME/cm3/bin vs. sharing binaries and having just split off config? What if they want to change the compiled code? To what extent are systems multi-user? So "yes" there is work to do here and we need help :) 2) Yes. Remember how I said I was lazy and made just one, or rather two, package sets? That is what you are seeing here. Olaf did put more thought/work into this and the .sh files do build a few different distribution sets. 3) I think so. Look for "www" in config? 1-3) yes. In particular, are you aware of the rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? I went with origin. I didn't have the energy or perhaps the buy-in to implement relink-upon-install approach. Previously we relied on either paths being correct *before* make install, or setting LD_LIBRARY_PATH. Both are bad solutions. There are I believe approx. 4 choices, for redistributable binaries that might go to a variety of setups: a) paths correct on original build machine b) LD_LIBARY_PATH c) origin d) relink upon install (leaving binaries not further relocatable by mv/cp). I went with c. So if you move samples around, you have to deal with that. bin/samples would seem easy enough though. Or heck, a peer of bin? lib? samplebin? samples? I think if we adopted automake/libtool, we'd get #c and it would be kinda good. Do cmake users use libtool? You know meta meta meta point -- we have our own little close to state of the art build system, but not quite state of the art. It works prety well and pretty portably. But not always as well as people would like. Between autoconf and Perl's method of port to every system, we are more like Perl, but also trying to find least common denominators that don't actually have to be ported. Does this make some sense? 4) Just command line parsing in make-dist.py. 5) yes 6) I don't understand. min is meant to be: 6a) a minimal toolset You can write hello world. cm3, cm3cg, mklib, config, m3core, libm3 text, threads not a gui This is subjective. The "resource" stuff should probably be here. 6b) It is meant to be enough to bootstrap another cm3 version from, older or newer. cm3 manages its dependencies carefully. It either carries things itself, or they are in min. For example, it doesn't use X Windows (or .i3 files describing it). upgrade.py/upgrade.sh assume min Now, granted, we can bootstrap from nothing as well. 7) You didn't ask. But I kinda think we should only have environment variables that start CM3 -- CM3 and start CM3_ actually. 8) Another large meta point -- this is all clearly imperfect, usually for lack of time or energy, not because we felt it was better this way. Though there is the angle I gave -- portability. Also, autoconf was probably less well established when most of this was written. The build/conf world seems still to be somewhat unsettled. 9) I have a larger unimplemented vision -- the system should be redistributed as one portable hard to read C or C++ .tar.gz. This doesn't work today, because "the frontend does layout" -- we essentially have 6 flavors of C: 1 posix 32bit little endian (x86, etc.) 2 posix 32bit big endian (ppc_darwin, sparc32) 3 posix 64bit little endian (amd64. etc.) 4 posix 64bit big endian (ppc64_darwin, sparc64) 5 Windows 32bit little endian (x86 mips PowerPC alpha, arm32, etc. ) 6 Windows 32bit big endian (hypothetical xbox 360? CE?) 7 Windows 64bit little endian (amd64, ia64, alpha64 etc.) 8 Windows 64bit big endian (hypothetical xbox 360?) We could today make 6 distributions of just C and bootstrap from one of them, picking the right one. Most 64bit systems can run 32bit, so strike those as bootstrap? But not quite, e.g. OpenBSD. I'd like the option for the frontend to defer layout to the backend, and for the C backend to output string expressions involving sizeof(size_t) or sizeof(char*). And, for win32 vs. posix, I'd like quake to compile both, and to generate a Makefile that picks the right one, or a .sh/.cmd. Oh, and jmpbuf size is in the generated C. I know how to fix that. bootstrap specifically could inflate it to the largest known case, which is quite large. Or, once we generate C++ for excpetion handling, jmpbuf should go away. - Jay > Date: Wed, 3 Jun 2015 08:17:49 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) > > On 6/2/2015 09:11, Jay K wrote: > > John, this and your other problems should all be fixed now. > > > > I was able to reproduce them mostly. > > I didn't actually reproduce the cvsup problem, but I see in the log you > > did set CM3, and I did test with that, and hit the cm3cg version > > mismatch, and the problem is pretty clear -- my mistake. > > > > I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && > > make-dist.py or skip right to make-dist.py. > > There was an additional problem skipping right to make-dist.py that I > > fixed, at least for Darwin. > > Hi Jay, > I set up the VM last night and it completed the make-dist.py command. I > have some questions / comments. > > comment 1: > Using bin for config isn't a standard unix location. > e.g. rather than /bin/config/AMD64_FREEBSD, it should be located > at /etc/modula3/config/AMD64_FREEBSD (can it be configurable?) > > comment 2: > Many of the binaries are might actually be example programs, e.g. cube, > calculator, fisheye. Rather than /bin where they risk clashing > with other programs, it might be advisable to stick them a > /share/examples/modula3 (configurable?). > > comment 3: /www is not standard either. We would be this in > /share/doc/modula3. Can it be configurable? > > Comments 1--3 might be related because www might have relative paths > that need changing > > comment 4: I'd like a configurable option not to tar/gzip the products. > In my case, I'm going to stage them manually so this is an unnecessary > step. > > comment 5: the "min" distribution seems suitable to be packaged as a > bootstrap > > question 6: In most use cases, other than intentionally recreating the > bootstrap, I'm not going to need the "min" distribution. Is there any > impact to building "all" only? In other words, does something else use > "min" distribution? > > Thanks for the update! > John > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 10:36:51 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 10:36:51 +0200 Subject: [M3devel] It works! In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st> Message-ID: <556EBCA3.3090601@marino.st> On 6/3/2015 09:18, Jay K wrote: > 1) "config"'s user configurability is over-implied imho. > [snip] > But also, we target not just Unix. > Yet we have almost the same directory structure on Unix and Windows. > The difference is that on Windows we put the .dlls in bin. It is possible this was misunderstood. It's important to release that cm3 will be installed nominally at /usr/local/bin. On a normal unix system, there could be over 500 packages all installing there. Things like configuration files (.conf) are nominally installed at /usr/local/etc/. The config directory is analogous to these. I will do the following in any case: A) move config from bin to etc// B) edit cm3.cfg to point to new location All I was suggesting is that the script do this for me. It can still install at bin/ by default, but it would be nice if it could install elsewhere too. > Should we use $HOME/etc or $HOME/config? Never. This does not work for packaging. E.g. prebuild binary packages that are installed (no building required) > 2) "example" progs > Yes. Remember how I said I was lazy and made just one, or rather two, > package sets? That is what you are seeing here. > Olaf did put more thought/work into this and the .sh files do build a > few different distribution sets. I am fine with what is built. Remember above that cm3 is installed with 500 other packages. Any program that is not essential and essentially there as an example could (and my opinion) should be installed in a place other than bin/. I don't care if it's installed outside of PATH because it probably won't be called much and absolute path is fine in this case. > 3) I think so. Look for "www" in config? > 1-3) yes. > In particular, are you aware of the > rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? I just remember that the current cm3 in ports, which is split up like I suggest, had some broken links because I moved these things around. I can't remember what is actually broken; I'd have to check again. The point was you can't just move stuff around without changing links, so the install script would have to do this if it supported alternative locations. > 4) > Just command line parsing in make-dist.py. Right, but currently I have to patch make-dist.py to make it not tar/gzip. If the script supported the option, I could avoid that patch. > 5) yes > 6) I don't understand. > min is meant to be: > 6a) a minimal toolset > You can write hello world. > cm3, cm3cg, mklib, config, m3core, libm3 > text, threads > not a gui > This is subjective. The "resource" stuff should probably be here. The port is only going to install one distribution, and that will be the "all". The non-used one is just a waste of building; it will be thrown away. Ports framework repackages and stored in FreeBSD's binary package format. > 7) You didn't ask. But I kinda think we should only have environment > variables > that start CM3 -- CM3 and start CM3_ actually. that sounds logical but as long as they are defined it doesn't really matter to me. > 8) Another large meta point -- this is all clearly imperfect, usually > for lack of time or energy, not because we felt it was better this way. > Though there is the angle I gave -- portability. > Also, autoconf was probably less well established when most of this was > written. > The build/conf world seems still to be somewhat unsettled. if a python script is sufficient for port needs, this issue isn't important to me. So far it seems that it is. > 9) I have a larger unimplemented vision -- the system should be > redistributed as one portable hard to read C or C++ .tar.gz. > This doesn't work today, because "the frontend does layout" -- we > essentially have 6 flavors of C: > We could today make 6 distributions of just C and bootstrap from one of > them, picking the right one. > Most 64bit systems can run 32bit, so strike those as bootstrap? But not > quite, e.g. OpenBSD. DragonFly is also 64-bit only. > I'd like the option for the frontend to defer layout to the backend, and > for the C backend to output string expressions involving sizeof(size_t) > or sizeof(char*). > And, for win32 vs. posix, I'd like quake to compile both, and to > generate a Makefile that picks the right one, or a .sh/.cmd. > Oh, and jmpbuf size is in the generated C. I know how to fix that. > bootstrap specifically could inflate it to the largest known case, which > is quite large. > Or, once we generate C++ for excpetion handling, jmpbuf should go away. no comment. :) John From jay.krell at cornell.edu Wed Jun 3 11:30:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 09:30:44 +0000 Subject: [M3devel] It works! In-Reply-To: <556EBCA3.3090601@marino.st> References: <556700B0.8060500@marino.st>, , , , ,,<556B4BE9.1000902@marino.st>, , , , ,,,,, , , , ,,,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , ,,,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , ,,, , , , , , , , <556C1218.3000103@marino.st> , , , , , <556C16E7.505@marino.st>, , , , ,,,,<556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st>, , <556EBCA3.3090601@marino.st> Message-ID: I believe people typically install to /usr/local/cm3 instead of /usr/local.There is a philosophical problem here in terms of setup file systemlayout. //package and then symlinks from /usr/local/{bin,lib}and removal of a package is just removing its one directory recursivelyand cleaning up any orphaned links vs. files put directly into/usr/local/{bin,lib} and data maintained on the side as to which filecame from which package. The first way gives more leeway to install extra stuff but not linkit into $PATH etc. The second way I believe is more common.It is easy to put together by make DESTDIR=whatever, tar.gz whateverand install is just untar, and uninstall either isn't implemented,or the data on the side is the output of tar tf, I guess. I put it in /cm3 on my systems and $HOME/cm3 where I don'thave that access, or $HOME/cm3. for filesystemsshared across targets -- e.g. the opencsw machines share filesystemacross sparc32, sparc64, x86, amd64. But yes, I understand. Just because it is called "config", does not make it "config".It should be refactored.And will there be some expectation that the "binaries" can beupdated while leaving "config" alone? > currently I have to patch make-dist.py to make it not > tar/gzip. If the script supported the option, I could avoid that patch. Send your patches here? Or fork on github and send pull requests? Another "less-binary" bootstrap option is the C archives.It is text files, but they are all generated and hard to read.I have to get the C backend back to working. :( Since you are going into ports, with an expected install place,and are building it yourself, you might want to mess withrpath, see here: jbook2:config-no-install jay$ pwd/dev2/cm3/m3-sys/cminstall/src/config-no-install book2:config-no-install jay$ grep rpath *ALPHA_OSF: args += [ "-rpath", LIB_USE ]FreeBSD.common: & " -Wl,-rpath,\\$ORIGIN"FreeBSD.common: & " -Wl,-rpath,\\$ORIGIN/../lib "Interix.common: % & " -Wl,-rpath,\\$ORIGIN"Interix.common: % & " -Wl,-rpath,\\$ORIGIN/../lib"Linux.common: & " -Wl,-rpath,\\$ORIGIN"Linux.common: & " -Wl,-rpath,\\$ORIGIN/../lib"NetBSD.common:M3_SHARED_LIB_ARG = "-Wl,-rpath-link,"OpenBSD.common:% & " -Wl,-rpath,\\$ORIGIN" % no $ORIGIN support up to/including 4.7OpenBSD.common:% & " -Wl,-rpath,\\$ORIGIN/../lib" % no $ORIGIN support up to/including 4.7OpenBSD.common:SYSTEM_LIBS{"ODBC"} = ["-Wl,-rpath,/usr/local/lib -L/usr/local/lib -liodbc" ]OpenBSD.common:SYSTEM_LIBS{"POSTGRES95"} = ["-Wl,-rpath -L/usr/local/lib -lpq"]gnuld.common: M3_SHARED_LIB_ARG = "-Wl,-rpath,"gnuld.common: GNU_LD_APPEND = " -Wl,-rpath," & INSTALL_ROOT & "/lib " % non-portablegnuld.common: M3_SHARED_LIB_ARG = "-Wl,-rpath," - Jay > Date: Wed, 3 Jun 2015 10:36:51 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] It works! > > On 6/3/2015 09:18, Jay K wrote: > > 1) "config"'s user configurability is over-implied imho. > > [snip] > > But also, we target not just Unix. > > Yet we have almost the same directory structure on Unix and Windows. > > The difference is that on Windows we put the .dlls in bin. > > It is possible this was misunderstood. > > It's important to release that cm3 will be installed nominally at > /usr/local/bin. On a normal unix system, there could be over 500 > packages all installing there. Things like configuration files (.conf) > are nominally installed at /usr/local/etc/. The config directory is > analogous to these. > > I will do the following in any case: > A) move config from bin to etc// > B) edit cm3.cfg to point to new location > > All I was suggesting is that the script do this for me. It can still > install at bin/ by default, but it would be nice if it could install > elsewhere too. > > > > > Should we use $HOME/etc or $HOME/config? > > Never. > This does not work for packaging. > E.g. prebuild binary packages that are installed (no building required) > > > > > 2) "example" progs > > Yes. Remember how I said I was lazy and made just one, or rather two, > > package sets? That is what you are seeing here. > > Olaf did put more thought/work into this and the .sh files do build a > > few different distribution sets. > > I am fine with what is built. Remember above that cm3 is installed with > 500 other packages. Any program that is not essential and essentially > there as an example could (and my opinion) should be installed in a > place other than bin/. I don't care if it's installed outside of PATH > because it probably won't be called much and absolute path is fine in > this case. > > > 3) I think so. Look for "www" in config? > > 1-3) yes. > > In particular, are you aware of the > > rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? > > > I just remember that the current cm3 in ports, which is split up like I > suggest, had some broken links because I moved these things around. I > can't remember what is actually broken; I'd have to check again. The > point was you can't just move stuff around without changing links, so > the install script would have to do this if it supported alternative > locations. > > > > 4) > > Just command line parsing in make-dist.py. > > Right, but currently I have to patch make-dist.py to make it not > tar/gzip. If the script supported the option, I could avoid that patch. > > > > 5) yes > > 6) I don't understand. > > min is meant to be: > > 6a) a minimal toolset > > You can write hello world. > > cm3, cm3cg, mklib, config, m3core, libm3 > > text, threads > > not a gui > > This is subjective. The "resource" stuff should probably be here. > > > The port is only going to install one distribution, and that will be the > "all". The non-used one is just a waste of building; it will be thrown > away. > > Ports framework repackages and stored in FreeBSD's binary package format. > > > 7) You didn't ask. But I kinda think we should only have environment > > variables > > that start CM3 -- CM3 and start CM3_ actually. > > that sounds logical but as long as they are defined it doesn't really > matter to me. > > > > 8) Another large meta point -- this is all clearly imperfect, usually > > for lack of time or energy, not because we felt it was better this way. > > Though there is the angle I gave -- portability. > > Also, autoconf was probably less well established when most of this was > > written. > > The build/conf world seems still to be somewhat unsettled. > > if a python script is sufficient for port needs, this issue isn't > important to me. So far it seems that it is. > > > 9) I have a larger unimplemented vision -- the system should be > > redistributed as one portable hard to read C or C++ .tar.gz. > > This doesn't work today, because "the frontend does layout" -- we > > essentially have 6 flavors of C: > > We could today make 6 distributions of just C and bootstrap from one of > > them, picking the right one. > > Most 64bit systems can run 32bit, so strike those as bootstrap? But not > > quite, e.g. OpenBSD. > > DragonFly is also 64-bit only. > > > I'd like the option for the frontend to defer layout to the backend, and > > for the C backend to output string expressions involving sizeof(size_t) > > or sizeof(char*). > > And, for win32 vs. posix, I'd like quake to compile both, and to > > generate a Makefile that picks the right one, or a .sh/.cmd. > > Oh, and jmpbuf size is in the generated C. I know how to fix that. > > bootstrap specifically could inflate it to the largest known case, which > > is quite large. > > Or, once we generate C++ for excpetion handling, jmpbuf should go away. > > no comment. :) > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 11:54:50 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 11:54:50 +0200 Subject: [M3devel] It works! In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , , <556C16E7.505@marino.st>, , , , , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st>, , <556EBCA3.3090601@marino.st> Message-ID: <556ECEEA.9040302@marino.st> On 6/3/2015 11:30, Jay K wrote: > I believe people typically install to /usr/local/cm3 instead of /usr/local. > There is a philosophical problem here in terms of setup file system > layout. //package and then symlinks from /usr/local/{bin,lib} > and removal of a package is just removing its one directory recursively > and cleaning up any orphaned links vs. files put directly into > /usr/local/{bin,lib} and data maintained on the side as to which file > came from which package. Well, setting it to /usr/local/cm3 would eliminate name clashes, but it would also move cm3 out of the standard path. That would require every user to adjust their path. If were were talking about 1-3 symlinks in /usr/local/bin then that would be fine, but if it's like 25+ then that could be kind of pointless. Plus on FreeBSD documentation, config, and examples go in standard locations (if possible) and setting to /usr/local/cm3 without modification wouldn't solve that. We aren't worried about orphan files or whatever, the binary package manager cleans up after itself when a package is removed, and won't let two versions of a package be installed simultaneously, so there's no real benefit to having cm3 in a dedicated directory (IMO). > Just because it is called "config", does not make it "config". > It should be refactored. > And will there be some expectation that the "binaries" can be > updated while leaving "config" alone? No. With updates, everything that was installed gets removed. If this is part of an update, everything is reinstalled. There's always integrity there. For BSD setups, having it at etc/modula3/config makes a lot of sense and there's no downside other than it's a different location from releases that you guys may make. > > currently I have to patch make-dist.py to make it not > > tar/gzip. If the script supported the option, I could avoid that patch. > > > Send your patches here? > Or fork on github and send pull requests? They wouldn't be suitable. They would be patches to make the script work meaning lines would be removed. You're thinking about a patch to handle both cases. > Since you are going into ports, with an expected install place, > and are building it yourself, you might want to mess with > rpath, see here: IIRC, the rpath was already fixed with the CM3 portable rpath option. I sort of recall hitting that before and finding $ORIGIN supported. If RPATH is wrong it definitely needs fixing but I think it is a problem that presents quickly (e.g. you couldn't use the product as a bootstrap). John From wagner at elegosoft.com Wed Jun 3 13:11:02 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 3 Jun 2015 13:11:02 +0200 Subject: [M3devel] It works! In-Reply-To: <556ECEEA.9040302@marino.st> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> Message-ID: <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> On Wed, 03 Jun 2015 11:54:50 +0200 John Marino wrote: > On 6/3/2015 11:30, Jay K wrote: > > I believe people typically install to /usr/local/cm3 instead of /usr/local. > > There is a philosophical problem here in terms of setup file system > > layout. //package and then symlinks from /usr/local/{bin,lib} > > and removal of a package is just removing its one directory recursively > > and cleaning up any orphaned links vs. files put directly into > > /usr/local/{bin,lib} and data maintained on the side as to which file > > came from which package. > > Well, setting it to /usr/local/cm3 would eliminate name clashes, but it > would also move cm3 out of the standard path. That would require every > user to adjust their path. > > If were were talking about 1-3 symlinks in /usr/local/bin then that > would be fine, but if it's like 25+ then that could be kind of pointless. > > Plus on FreeBSD documentation, config, and examples go in standard > locations (if possible) and setting to /usr/local/cm3 without > modification wouldn't solve that. > > We aren't worried about orphan files or whatever, the binary package > manager cleans up after itself when a package is removed, and won't let > two versions of a package be installed simultaneously, so there's no > real benefit to having cm3 in a dedicated directory (IMO). > > > Just because it is called "config", does not make it "config". > > It should be refactored. > > And will there be some expectation that the "binaries" can be > > updated while leaving "config" alone? > > No. With updates, everything that was installed gets removed. If this > is part of an update, everything is reinstalled. There's always > integrity there. For BSD setups, having it at etc/modula3/config makes > a lot of sense and there's no downside other than it's a different > location from releases that you guys may make. It doesn't really fit, but never mind. What is different about M3 is that it contains both the base development system and dozens of utility and application packages, and every user is supposed to be able to update and install packages (only it's called shipping, not installation). Theoretically, you could package each M3 package in its own BSD package; I even think there was one (historical) distribution of PM3 that did exactly that. That doesn't address the problem, as each user needs to become root to ship a package. As I've written in another m3 thread yesterday, what would really be needed is a multi-level package pool hierarchy and an integrated m3 builder that copes with that and sets of packages. But even if we had that, I'm not sure how it would fit into the standard Unix file system hierarchy layout: put the user-managed package pools under /var/local/lib? As the package systems vary significantly between all the supported platforms of M3, the concepts of M3 packages and OS packages have never been unified. And of course the M3 project never had enough personal resources to provide OS specific packages for even a small subset of the target platforms. Perhaps a pragmatical way to handle this would be to package and install a minimal system only containing the builder, compiler and base libraries as an OS package in an OS-specific way (including documentation and all) and add support for the standard M3 package hierarchy under /usr/local/cm3 (group-writable by an m3 group). It would be a compromise, but perhaps a better one than installing the complete CM3 distribution as one big package read-only for the ordinary user. The CM3 package pool under /usr/local/cm3 could even be integrated with the Github repository and allow easy checkout of all M3 packages there and even pushing new branches and packages. The builder could locate, checkout and install missing imported packages, have an option for creating a new package template and a standard way to publish this on Gibhub. This might help to lower the threshold for new users to become active in the community and share their code. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From adacore at marino.st Wed Jun 3 13:46:07 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 13:46:07 +0200 Subject: [M3devel] It works! In-Reply-To: <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> Message-ID: <556EE8FF.3060502@marino.st> On 6/3/2015 13:11, Olaf Wagner wrote: > What is different about M3 is that it contains both the base development > system and dozens of utility and application packages, and every user > is supposed to be able to update and install packages (only it's > called shipping, not installation). If I just install the "all" distribution, as built, in /usr/local/cm3 (with the possible exception of shifting www to standard docs location): 1) It is a read-only area (ideally) 2) It would require all users to set /usr/local/cm3 in PATH. This is not really that big of a deal. It's not the first package to require something like that, plus, frankly, the m3 community is not large and such a requirement could be imposed without a lot of backlash. Maybe this is the best solution after all (and again it makes the port makefile simpler) > Theoretically, you could package each M3 package in its own BSD > package; I even think there was one (historical) distribution of PM3 > that did exactly that. That doesn't address the problem, as each user > needs to become root to ship a package. Breaking each into it's own package is what should happen, but that requires work on my part that I'm not able to give, so everyone has to deal with the giant port. If m3 regains enough popularity to justify splitting into individual packages then we can revisit the decision. > As I've written in another m3 thread yesterday, what would really > be needed is a multi-level package pool hierarchy and an integrated > m3 builder that copes with that and sets of packages. But even if we had > that, I'm not sure how it would fit into the standard Unix file system > hierarchy layout: put the user-managed package pools under /var/local/lib? At least for ports, packages either go in root's areas (e.g. /usr/local) or user-accessible areas but not mixed. One can install M3 in a user's area and do what they want. But a generic user can't install to standard areas like /usr/local and most /var areas. > As the package systems vary significantly between all the supported > platforms of M3, the concepts of M3 packages and OS packages have > never been unified. And of course the M3 project never had enough > personal resources to provide OS specific packages for even a small > subset of the target platforms. Pragmatically, you want the individual platforms to package it for you. Less work for you and it's probably done more correctly. E.g. m3 is self-hosting on FreeBSD so there's no real need to provide FreeBSD support now (with the possible exception of a statically linked "min" distribution) Ideally, similar support occurs on Debian, Fedora, Arch, macports, etc. John From adacore at marino.st Wed Jun 3 13:57:38 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 13:57:38 +0200 Subject: [M3devel] Are all 5 gcc branches used? Message-ID: <556EEBB2.30809@marino.st> I see 5 gcc branches in m3-sys/m3cc. I've seen gcc-4.7 in use and I can guess gcc-apple is still used, but are gcc, gcc-4.5, and gcc-4.6 branches obsolete? There reason why it matters is that I'm using github's built-in "create tarball from top of repo" capability as the digest-confirmed source distribution file. All the extra gcc branches are causing the compressed tarball to be pretty big. Currently it's at 150Mb. If some of these gcc branches are unused and will never again be used, I might suggest they are pruned. It's in the repository so they could be retrieved with the right hash, and it would dramatically reduce the size of the archive tarball. If all 5 versions of gcc are still used then that's a horse of a different color. John From jay.krell at cornell.edu Wed Jun 3 18:36:17 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 3 Jun 2015 09:36:17 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: > Other than cross-compile from 32 to 64 bit what purpose does this serve? > > Sent from my iPad > > On Jun 2, 2015, at 9:58 PM, Jay K wrote: > >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 >> index fa72589..37bf238 100644 >> --- a/m3-libs/m3core/src/text/TextLiteral.i3 >> +++ b/m3-libs/m3core/src/text/TextLiteral.i3 >> @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; >> IMPORT RTHooks, TextClass; >> >> CONST >> - (* DIV BITSIZE should not be here! *) >> +(* When cm3 uses TInt and when pickle special cases this, it should be approx: >> + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) >> +(* This fails for 32bit host and 64bit target: >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) >> +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) >> + dependent. *) >> + (* DIV BITSIZE should not be here -- compiler limitation due to >> + non-use of TInt. *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; >> (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) >> (* Do not adjust this for INTEGER size. It makes T have different >> fingerprints on 32- and 64-bit machines, which undermines pickling/ >> >> >> Yes I know TEXT pickles will be broken. >> I propose that unpickle should special case a few values here. >> Or, isn't the brand supposed to help? >> Or, can we encode this in a better way, w/o an upper bound? >> I know if you put in 0..0 or 0..-1 you incur range violations at runtime. >> Which makes me wonder -- are TEXTs unsafe? >> >> Should we support a syntax something like: >> >> cnt : INTEGER; >> buf : ARRAY [0..cnt - 1] OF Byte; >> ? >> >> Thanks, >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 18:39:46 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 3 Jun 2015 09:39:46 -0700 Subject: [M3devel] It works! In-Reply-To: <556EE8FF.3060502@marino.st> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> <556EE8FF.3060502@marino.st> Message-ID: <5649AA84-7127-49E1-9F80-7E3F425A2DE5@gmail.com> I am sympathetic to "path pollution" concerns even if in usr/local/cm3. We can move some stuff to demobin or samplebin if agreement on uselessness. And/or prefix every bin & lib with m3 or cm3. We should also rev our so versions? & and install version-named cm3 & cm3cg? - Jay On Jun 3, 2015, at 4:46 AM, John Marino wrote: > On 6/3/2015 13:11, Olaf Wagner wrote: >> What is different about M3 is that it contains both the base development >> system and dozens of utility and application packages, and every user >> is supposed to be able to update and install packages (only it's >> called shipping, not installation). > > If I just install the "all" distribution, as built, in /usr/local/cm3 > (with the possible exception of shifting www to standard docs location): > > 1) It is a read-only area (ideally) > 2) It would require all users to set /usr/local/cm3 in PATH. > > This is not really that big of a deal. It's not the first package to > require something like that, plus, frankly, the m3 community is not > large and such a requirement could be imposed without a lot of backlash. > > Maybe this is the best solution after all (and again it makes the port > makefile simpler) > > >> Theoretically, you could package each M3 package in its own BSD >> package; I even think there was one (historical) distribution of PM3 >> that did exactly that. That doesn't address the problem, as each user >> needs to become root to ship a package. > > Breaking each into it's own package is what should happen, but that > requires work on my part that I'm not able to give, so everyone has to > deal with the giant port. If m3 regains enough popularity to justify > splitting into individual packages then we can revisit the decision. > >> As I've written in another m3 thread yesterday, what would really >> be needed is a multi-level package pool hierarchy and an integrated >> m3 builder that copes with that and sets of packages. But even if we had >> that, I'm not sure how it would fit into the standard Unix file system >> hierarchy layout: put the user-managed package pools under /var/local/lib? > > At least for ports, packages either go in root's areas (e.g. /usr/local) > or user-accessible areas but not mixed. One can install M3 in a user's > area and do what they want. But a generic user can't install to > standard areas like /usr/local and most /var areas. > > >> As the package systems vary significantly between all the supported >> platforms of M3, the concepts of M3 packages and OS packages have >> never been unified. And of course the M3 project never had enough >> personal resources to provide OS specific packages for even a small >> subset of the target platforms. > > Pragmatically, you want the individual platforms to package it for you. > Less work for you and it's probably done more correctly. E.g. m3 is > self-hosting on FreeBSD so there's no real need to provide FreeBSD > support now (with the possible exception of a statically linked "min" > distribution) > > Ideally, similar support occurs on Debian, Fedora, Arch, macports, etc. > > John > > From jay.krell at cornell.edu Wed Jun 3 19:49:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 17:49:07 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> Message-ID: I don't have any, but should we have such artificial limits? I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." And even that isn't right -- 4bg is often not lenty for a 32bit host. File sizes exceed that long ago.But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. Given that a blueray movie is an array of bytes, is a text inappropriate? The actual diff presented just adjusts the limit by 8 bytes. Ok? Or must preserve pickles and can't do this w/o some other change? This isn't a more general problem? Or usually pickles are 32/64-compatible, but this is a most unusual case? Thanks, - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 13:24:36 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Why would you ever want a TEXT literal that is so large?By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface.A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. On Jun 3, 2015, at 12:36 PM, Jay wrote:I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: Other than cross-compile from 32 to 64 bit what purpose does this serve? Sent from my iPad On Jun 2, 2015, at 9:58 PM, Jay K wrote: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 20:04:04 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 18:04:04 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556EEBB2.30809@marino.st> References: <556EEBB2.30809@marino.st> Message-ID: They likely aren't all used.I'd like to make none of them used, but that is another matter. Look in src/m3makefile. I think OpenBSD targets might be back on gcc-4.2.There is flexibility in there, in m3makefile and "parse.c", but maybe at too high cost. The problem, you can imagine, is that Apple has as fork, OpenBSD has a fork, we have a fork, and there is mainline.Ideally everyone would be in mainline and nobody would have any forks. As we/I move from major version to version, I generally make a new fork. I have a certain paranoia and laziness when debugging -- "does it work with the old one" -- "how easy is to reconstruct the old one? Oh, cool, it is still right there, I don't have to figure out how to go backwards in source control, and I can easily have them both side by side". I realize source control could help me here. It also allows for "staging", i.e. I can bring in new version, test some targets, but leave the other targets alone, waiting for myself or others to test them later. Or, decide they are little enough used just move them forward w/o testing. If gcc obsoletes targets we want to keep, we could keep the old versions. Not super useful given our usage levels. We could maybe also minimize our changes and distribute patches only.Sometimes maybe I went overboard with my changes. Or we could ditch gcc entirely and use the C backend and/or LLVM, hopefully both unpatched. :) Try xz instead of gzip, maybe it halves the size? - Jay > Date: Wed, 3 Jun 2015 13:57:38 +0200 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Are all 5 gcc branches used? > > I see 5 gcc branches in m3-sys/m3cc. I've seen gcc-4.7 in use and I can > guess gcc-apple is still used, but are gcc, gcc-4.5, and gcc-4.6 > branches obsolete? > > There reason why it matters is that I'm using github's built-in "create > tarball from top of repo" capability as the digest-confirmed source > distribution file. All the extra gcc branches are causing the > compressed tarball to be pretty big. Currently it's at 150Mb. > > If some of these gcc branches are unused and will never again be used, I > might suggest they are pruned. It's in the repository so they could be > retrieved with the right hash, and it would dramatically reduce the size > of the archive tarball. > > If all 5 versions of gcc are still used then that's a horse of a > different color. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 20:10:07 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 20:10:07 +0200 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st> Message-ID: <556F42FF.9000106@marino.st> On 6/3/2015 20:04, Jay K wrote: > If gcc obsoletes targets we want to keep, we could keep the old > versions. Not super useful given our usage levels. It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not obsolete by that definition, so all 5 of these are obsolete. I would definitely encourage to prune as many of these as possible. I could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" based on the base compiler, why are you assuming it must be built by base compiler? By definition, the base compiler is only required to build base. There are much newer and well maintained versions of gcc in openbsd ports tree. There's no reason a ports compiler couldn't be used (I assume this is actually common). > Try xz instead of gzip, maybe it halves the size? I can't influence github's API. It is what it is. John From hendrik at topoi.pooq.com Wed Jun 3 21:07:46 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 3 Jun 2015 15:07:46 -0400 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st> Message-ID: <20150603190745.GA11809@topoi.pooq.com> On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > hopefully both unpatched. :) Weren't you working on a C backend for Modula a year or two ago? What happened to it? Is it available for use? If so, where and how? -- hendrik From rodney_bates at lcwb.coop Wed Jun 3 21:23:07 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 03 Jun 2015 14:23:07 -0500 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: References: Message-ID: <556F541B.5080207@lcwb.coop> On 06/02/2015 09:51 PM, Jay K wrote: > Ideally: > 1 32bit host could target 64bit target > 2 32bit texts could be as large as the address space/OS allows. > 3 ditto 64bit texts -- larger than 4GB > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > Currently only #4 is true. > TEXTs for any target can only be about 500MB. > > > To fix 2 and 3 requires m3front to use TInt more. > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > Can we special case somehow? > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > Should/can we introduce one? > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > index fa72589..4d16a44 100644 > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > CONST > (* DIV BITSIZE should not be here! *) > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; I'd say go ahead and change this. We really do want it to cross-compile sooner than anybody will be able to put TInt in all over the place. I did not build a cross compiler, but did what I think should be a realistic simulation of it by changing the cnt:INTEGER field to two INTEGER fields and compiling with a 32-bit compiler. That would be the same size as a 64-bit INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to compile, with bit size having space for 31 more bits before it overflows. This byte count of buf is a multiple of 4, which we want for unicode-range WIDECHAR. Making it -10 overflows, presumably because the additional byte is padded out to a multiple of 32 bits. I would have expected that somewhere, a size including the one-word heap header would be computed, but these are never actually heap allocated, and it seems to work. But to hedge against things, I suggest using -19. This would allow space to count it, cross-compiling 32->64. This will allow a text literal of 268435436 ISO characters, and a wide text literal of 1/4 that, which is not likely to be a serious limitation, considering that it all has to be on a single line of source code. As for pickles, the horse is already out of the barn on that, as pickles could already have been written with the current fingerprint, and we can't change the type in any way without the fingerprint changing. It is not hard to add recognition of the old fingerprint to pickles. Right off hand, I don't remember the process for finding actual fingerprint values, but I've done it before and can rediscover it. > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > (* Do not adjust this for INTEGER size. It makes T have different > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > otherwise we have: > > > new source -> compiling TextLiteral.i3 > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > 1 error encountered > > > Ok? > - Jay -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Jun 3 21:29:24 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 03 Jun 2015 14:29:24 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <556F5594.7070701@lcwb.coop> On 06/02/2015 08:58 PM, Jay K wrote: > We cannot cross from 32bit host to 64bit target. > > > Ok to commit this? > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > index fa72589..37bf238 100644 > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; > IMPORT RTHooks, TextClass; > CONST > - (* DIV BITSIZE should not be here! *) > +(* When cm3 uses TInt and when pickle special cases this, it should be approx: > + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) > +(* This fails for 32bit host and 64bit target: > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) > +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) > + dependent. *) > + (* DIV BITSIZE should not be here -- compiler limitation due to > + non-use of TInt. *) > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > (* Do not adjust this for INTEGER size. It makes T have different > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > Yes I know TEXT pickles will be broken. > I propose that unpickle should special case a few values here. > Or, isn't the brand supposed to help? > Or, can we encode this in a better way, w/o an upper bound? > I know if you put in 0..0 or 0..-1 you incur range violations at runtime. > Which makes me wonder -- are TEXTs unsafe? > > Should we support a syntax something like: > > cnt : INTEGER; > buf : ARRAY [0..cnt - 1] OF Byte; > ? > Can't do this. Non-open arrays must have static constant bounds, which the above does not. Open arrays have extra dope and are accessed through two extra pointers, which we really wouldn't want for text literals. TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that anything else in UNSAFE code. The interface is UNSAFE, which means safe code can't IMPORT it, and thus can't see the declaration. Not very likely that anybody other than the runtime, compiler-generated code, and debugger (written in C anyway) would want to mess with it. > Thanks, > - Jay -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 4 00:55:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 22:55:00 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556F42FF.9000106@marino.st> References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: > obsolete I meant, gcc 4.9 might not target Alpha/OSF, but gcc 4.5 might. > OpenBSD gcc 4.2. True -- we could take the OpenBSD port of gcc 4.7 or such and apply those patches. In fact... 1) most of the patches are to the driver, that we don't use Things like building with #define __OpenBSD__ and making size_t == unsigned long or unsigned int. 2) Maybe I'm doing this. I recall checking patch files and applying them either at build time or commiting those. 3) Really, you know, from a compiler backend point of view, OpenBSD, NetBSD, FreeBSD, all the same thing -- same ABI, same x86 instructions, same object file format. Various targets collapse down to fewer. So we don't always need the patches. But then something like ARM_DARWIN, which I never got fully working, those might be some more serious patches. 4.2 might also be Apple or Interix related. I'll have to look. We could probably also getting away with such things as: OpenBSD, Interix, ARM_DARWIN: C backend only, if even that I do have to restore it to working. It was totally working. Anyway, point taken/reminded -- we should see about pruning. I just get nervous, you know, I'm not a CVS or git expert -- how do I get back the old stuff? > xz vs. gzip Does the ports system now download from github? Cool. gzip is much faster for compression. - Jay > Date: Wed, 3 Jun 2015 20:10:07 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On 6/3/2015 20:04, Jay K wrote: > > If gcc obsoletes targets we want to keep, we could keep the old > > versions. Not super useful given our usage levels. > > It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not > obsolete by that definition, so all 5 of these are obsolete. > > I would definitely encourage to prune as many of these as possible. I > could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" > based on the base compiler, why are you assuming it must be built by > base compiler? > > By definition, the base compiler is only required to build base. There > are much newer and well maintained versions of gcc in openbsd ports > tree. There's no reason a ports compiler couldn't be used (I assume > this is actually common). > > > Try xz instead of gzip, maybe it halves the size? > > I can't influence github's API. It is what it is. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 00:58:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 22:58:50 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <20150603190745.GA11809@topoi.pooq.com> References: <556EEBB2.30809@marino.st>, , <20150603190745.GA11809@topoi.pooq.com> Message-ID: It was totally working. I built the whole Darwin x86/amd64 system, ran gui apps.Got AMD64_NT working.Built all four variations of Solaris I think.Maybe cross bootstrapped but didn't c_compile other systems.I386_NT was probably working. And now if you try it there are assertion failures, like about right shiftinga signed type. Some errors about duplicate typedefs. It has bit-rotted slightly. Maybe with the Unicode work? Anyway, I'll repair it. It really was fully working.Debugging was getting better, better than non-m3gdb systems.There is still a uniquifier appended to every local.In case you have; int a;{ int a;{ int a;}}} we give youint a_1;int a_2;int a_3; I need to fix it to only uniqueify in the presence of duplicates. Globals should be made debuggable instead of just being an array of bytes.Other than globals, structs have fields.TEXTs aren't viewable I think. But for systems like Darwin and NT and HP-UX that don't support m3gdb, debugging is already better.Build times are noticeably regressed. - Jay > Date: Wed, 3 Jun 2015 15:07:46 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > > hopefully both unpatched. :) > > Weren't you working on a C backend for Modula a year or two ago? > What happened to it? Is it available for use? If so, where and how? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:04:43 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:04:43 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st>, , , , <20150603190745.GA11809@topoi.pooq.com>, Message-ID: Once it is working, the way you use it is one of: ./do-cm3-all.py c buildship or change M3_BACKEND_MODE to "C" in the config file. and it handles nested functions even... You can bootstrap with./boot1.py c which gives you a directory of c files and a makefile.cd there and make or nmake. But let me fix it first. It got broken recently. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Wed, 3 Jun 2015 22:58:50 +0000 Subject: Re: [M3devel] Are all 5 gcc branches used? It was totally working. I built the whole Darwin x86/amd64 system, ran gui apps.Got AMD64_NT working.Built all four variations of Solaris I think.Maybe cross bootstrapped but didn't c_compile other systems.I386_NT was probably working. And now if you try it there are assertion failures, like about right shiftinga signed type. Some errors about duplicate typedefs. It has bit-rotted slightly. Maybe with the Unicode work? Anyway, I'll repair it. It really was fully working.Debugging was getting better, better than non-m3gdb systems.There is still a uniquifier appended to every local.In case you have; int a;{ int a;{ int a;}}} we give youint a_1;int a_2;int a_3; I need to fix it to only uniqueify in the presence of duplicates. Globals should be made debuggable instead of just being an array of bytes.Other than globals, structs have fields.TEXTs aren't viewable I think. But for systems like Darwin and NT and HP-UX that don't support m3gdb, debugging is already better.Build times are noticeably regressed. - Jay > Date: Wed, 3 Jun 2015 15:07:46 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > > hopefully both unpatched. :) > > Weren't you working on a C backend for Modula a year or two ago? > What happened to it? Is it available for use? If so, where and how? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:14:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:14:49 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> Message-ID: sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched.An array of chars? Probably has same problem. Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 14:07:02 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu It?s only TEXT literals that are limited here. As in, what appears in a source program. On Jun 3, 2015, at 1:49 PM, Jay K wrote:I don't have any, but should we have such artificial limits? I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." And even that isn't right -- 4bg is often not lenty for a 32bit host. File sizes exceed that long ago.But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. Given that a blueray movie is an array of bytes, is a text inappropriate? The actual diff presented just adjusts the limit by 8 bytes. Ok? Or must preserve pickles and can't do this w/o some other change? This isn't a more general problem? Or usually pickles are 32/64-compatible, but this is a most unusual case? Thanks, - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 13:24:36 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Why would you ever want a TEXT literal that is so large?By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface.A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. On Jun 3, 2015, at 12:36 PM, Jay wrote:I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: Other than cross-compile from 32 to 64 bit what purpose does this serve? Sent from my iPad On Jun 2, 2015, at 9:58 PM, Jay K wrote: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:18:11 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:18:11 +0000 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: <556F541B.5080207@lcwb.coop> References: , <556F541B.5080207@lcwb.coop> Message-ID: Should we just be a little lax and say minus 64?Or pick some other number like 256MB? Can we declare there is no bound? Thank you, - Jay > Date: Wed, 3 Jun 2015 14:23:07 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) > > > > On 06/02/2015 09:51 PM, Jay K wrote: > > Ideally: > > 1 32bit host could target 64bit target > > 2 32bit texts could be as large as the address space/OS allows. > > 3 ditto 64bit texts -- larger than 4GB > > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > > Currently only #4 is true. > > TEXTs for any target can only be about 500MB. > > > > > > To fix 2 and 3 requires m3front to use TInt more. > > > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > > Can we special case somehow? > > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > > Should/can we introduce one? > > > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > > index fa72589..4d16a44 100644 > > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > > CONST > > (* DIV BITSIZE should not be here! *) > > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > > I'd say go ahead and change this. We really do want it to cross-compile > sooner than anybody will be able to put TInt in all over the place. > > I did not build a cross compiler, but did what I think should be a realistic > simulation of it by changing the cnt:INTEGER field to two INTEGER fields and > compiling with a 32-bit compiler. That would be the same size as a 64-bit > INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to > compile, with bit size having space for 31 more bits before it overflows. > This byte count of buf is a multiple of 4, which we want for unicode-range > WIDECHAR. Making it -10 overflows, presumably because the additional byte is > padded out to a multiple of 32 bits. > > I would have expected that somewhere, a size including the one-word heap header > would be computed, but these are never actually heap allocated, and it seems to > work. But to hedge against things, I suggest using -19. This would allow space > to count it, cross-compiling 32->64. > > This will allow a text literal of 268435436 ISO characters, and a wide text literal > of 1/4 that, which is not likely to be a serious limitation, considering that it > all has to be on a single line of source code. > > As for pickles, the horse is already out of the barn on that, as pickles could > already have been written with the current fingerprint, and we can't change the > type in any way without the fingerprint changing. It is not hard to add > recognition of the old fingerprint to pickles. Right off hand, I don't remember > the process for finding actual fingerprint values, but I've done it before and > can rediscover it. > > > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > > (* Do not adjust this for INTEGER size. It makes T have different > > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > > > > > otherwise we have: > > > > > > new source -> compiling TextLiteral.i3 > > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > > 1 error encountered > > > > > > Ok? > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Thu Jun 4 16:01:37 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Thu, 04 Jun 2015 16:01:37 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <55705A41.7070503@elstel.org> Am 03.06.15 um 03:58 schrieb Jay K: > We cannot cross from 32bit host to 64bit target. > > > Ok to commit this? > > Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. To me this is just another argument for a language specified value range of INTEGER. It makes certain things easier. My suggestion was: TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) BITS 64 FOR INTEGER would only work on x64 systems. The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same tends to be much slower). Regards, Elmar > Can't do this. Non-open arrays must have static constant bounds, > which the above does not. Open arrays have extra dope and are accessed > through two extra pointers, which we really wouldn't want for text > literals. > > TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that > anything > else in UNSAFE code. The interface is UNSAFE, which means safe code > can't IMPORT > it, and thus can't see the declaration. Not very likely that anybody > other than > the runtime, compiler-generated code, and debugger (written in C > anyway) would > want to mess with it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 4 16:07:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 09:07:48 -0500 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: References: , <556F541B.5080207@lcwb.coop> Message-ID: <55705BB4.4090408@lcwb.coop> On 06/03/2015 06:18 PM, Jay K wrote: > Should we just be a little lax and say minus 64? > Or pick some other number like 256MB? > Sure, just to allow for unexpected changes. My principle here was to make it as large as possible, subject to accommodating both native word sizes, in pickling, compiling (including cross compiling), etc. Plus, if it changes, then pickles need to be adapted, so I hope to get it to something we can keep. Do subtract a number that is one less than a multiple of 4, so MaxBytes comes out a multiple of 4. This is pretty confusing. What is really wanted is MaxBytes = (2^31) DIV BITSIZE (Byte) - , but I'm not sure compile-time DIV for this overflow case is well-defined, so 16_7FFFFFFF DIV BITSIZE(Byte) + 1 gives the max multiple of 4 bytes whose bit size is <= LAST(Int32). Then subtract another multiple of 4. So maybe 16_7FFFFFFF DIV BITSIZE(Byte) + 1 - 64 is easier to understand. > Can we declare there is no bound? > No, we're writing code that always ignores the type's bound anyway and uses the cnt field instead. We just want the type's bound as high as possible, to minimize the coding trouble of avoiding the compiler's checks. This is just the kind of low-level coding that the language's safe subset is designed to disallow. > Thank you, > - Jay > > > > Date: Wed, 3 Jun 2015 14:23:07 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) > > > > > > > > On 06/02/2015 09:51 PM, Jay K wrote: > > > Ideally: > > > 1 32bit host could target 64bit target > > > 2 32bit texts could be as large as the address space/OS allows. > > > 3 ditto 64bit texts -- larger than 4GB > > > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > > > Currently only #4 is true. > > > TEXTs for any target can only be about 500MB. > > > > > > > > > To fix 2 and 3 requires m3front to use TInt more. > > > > > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > > > Can we special case somehow? > > > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > > > Should/can we introduce one? > > > > > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > > > index fa72589..4d16a44 100644 > > > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > > > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > > > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > > > CONST > > > (* DIV BITSIZE should not be here! *) > > > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > > > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > > > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > > > > I'd say go ahead and change this. We really do want it to cross-compile > > sooner than anybody will be able to put TInt in all over the place. > > > > I did not build a cross compiler, but did what I think should be a realistic > > simulation of it by changing the cnt:INTEGER field to two INTEGER fields and > > compiling with a 32-bit compiler. That would be the same size as a 64-bit > > INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to > > compile, with bit size having space for 31 more bits before it overflows. > > This byte count of buf is a multiple of 4, which we want for unicode-range > > WIDECHAR. Making it -10 overflows, presumably because the additional byte is > > padded out to a multiple of 32 bits. > > > > I would have expected that somewhere, a size including the one-word heap header > > would be computed, but these are never actually heap allocated, and it seems to > > work. But to hedge against things, I suggest using -19. This would allow space > > to count it, cross-compiling 32->64. > > > > This will allow a text literal of 268435436 ISO characters, and a wide text literal > > of 1/4 that, which is not likely to be a serious limitation, considering that it > > all has to be on a single line of source code. > > > > As for pickles, the horse is already out of the barn on that, as pickles could > > already have been written with the current fingerprint, and we can't change the > > type in any way without the fingerprint changing. It is not hard to add > > recognition of the old fingerprint to pickles. Right off hand, I don't remember > > the process for finding actual fingerprint values, but I've done it before and > > can rediscover it. > > > > > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > > > (* Do not adjust this for INTEGER size. It makes T have different > > > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > > > > > > > > > otherwise we have: > > > > > > > > > new source -> compiling TextLiteral.i3 > > > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > > > 1 error encountered > > > > > > > > > Ok? > > > - Jay > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Jun 4 16:37:06 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 09:37:06 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> Message-ID: <55706292.9060608@lcwb.coop> On 06/03/2015 06:14 PM, Jay K wrote: > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > An array of chars? Probably has same problem. > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > - Jay > I'm OK with doing it, but I thought you had been talking about using TInt. Would that be better? It would be more general. But LONGINT would be faster at compile time, and less work, since the arithmetic operators would not need to be changed to TInt function calls. There might be quite a lot of those I guess the compiler would be of about equal help in finding missed places to be changed, either way. I just checked, and LONGINT is in the release compiler, contrary to my sense of relative history, so there would be no bootstrap barrier. Either would allow a 32-bit hosted compiler (cross- or native-) to handle types whose byte-count approached 2^31, instead of just 2^23. With a 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, whereas LONGINT would only go to 2^55. Do we really care? > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > From: hosking at purdue.edu > Date: Wed, 3 Jun 2015 14:07:02 -0400 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 4 17:08:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:08:38 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55706292.9060608@lcwb.coop> References: , , , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, , <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , , , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, , <55706292.9060608@lcwb.coop> Message-ID: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:41:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:41:47 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55705A41.7070503@elstel.org> References: , <55705A41.7070503@elstel.org> Message-ID: Do you want 64bit targets to be "better" or "the same"?What if I say the Modula-3 equivalent of, three programs: 1 printf("%d", (int)sizeof(void*)); 2 printf("%d", (int)sizeof(size_t)); 3 printf("%d", (int)sizeof(int)); One could argue in your favor here.The user of void* and size_t is expecting "better".The user of int is expecting "same". Modula-3 could have two integer types, builtin, short names.Or, today, it offers all 4 sizes, you just have to go slightly out of your way. You can say: TYPE int = Ctypes.int; or maybe BasicCtypes.int. and go about your business using int instead of INTEGER andget the compability you desire. Perhaps this should be: int = IntegerTypes.INT32? to not be be C-related? - Jay Date: Thu, 4 Jun 2015 16:01:37 +0200 From: estellnb at elstel.org To: jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem Am 03.06.15 um 03:58 schrieb Jay K: We cannot cross from 32bit host to 64bit target. Ok to commit this? Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. To me this is just another argument for a language specified value range of INTEGER. It makes certain things easier. My suggestion was: TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) BITS 64 FOR INTEGER would only work on x64 systems. The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same tends to be much slower). Regards, Elmar Can't do this. Non-open arrays must have static constant bounds, which the above does not. Open arrays have extra dope and are accessed through two extra pointers, which we really wouldn't want for text literals. TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that anything else in UNSAFE code. The interface is UNSAFE, which means safe code can't IMPORT it, and thus can't see the declaration. Not very likely that anybody other than the runtime, compiler-generated code, and debugger (written in C anyway) would want to mess with it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:45:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:45:00 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, <55706292.9060608@lcwb.coop>, , <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: How strongly preferred and why? good naturedly: 1) to cause me pain so that might give up? 2) to take more time so I don't muck with other stuff? 3) for compatibility with older releases? 4) for easier extension in future? 5) to cause me pain so I whine more about wanting operator overloading? I started this once and it going to be a pain. LONGINT is probably much easier. Maybe I have more patience now. Extension in the future could be addition of INTEGER128 to the language. :) And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128and convert frontend to use it. Hypothetically. Perhaps before that happenswe'll have operator overloading. :) - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 11:32:48 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu Again, TInt preferred. Sent from my iPhone On Jun 4, 2015, at 11:08 AM, Jay K wrote: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:54:51 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 4 Jun 2015 08:54:51 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <3A400392-6754-445E-B64F-7E8F1F47985D@purdue.edu> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <3A400392-6754-445E-B64F-7E8F1F47985D@purdue.edu> Message-ID: <119B30EC-1C76-41FB-BCBF-378E287479EF@gmail.com> > I don?t know what the static limit is on an array literal. I suspect all sizes and offsets are limited to bit counts stored INTEGER. So any field or type can be 512mb-1 max on 32bit hosted compiler and 2^56-1 for 64bit hosted compiler. How about a syntax that omits the end of the range? Type t = array [0..] of char? - Jay On Jun 3, 2015, at 5:03 PM, Antony Hosking wrote: > It would not make sense to store an entire blueray movie as a literal?better to structure it. > I don?t know what the static limit is on an array literal. > >> On Jun 3, 2015, at 7:14 PM, Jay K wrote: >> >> sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. >> An array of chars? Probably has same problem. >> >> >> Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? >> >> - Jay >> >> >> >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Wed, 3 Jun 2015 14:07:02 -0400 >> CC: m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> It?s only TEXT literals that are limited here. As in, what appears in a source program. >> >> On Jun 3, 2015, at 1:49 PM, Jay K wrote: >> >> I don't have any, but should we have such artificial limits? >> I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". >> Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." >> And even that isn't right -- 4bg is often not lenty for a 32bit host. >> File sizes exceed that long ago. >> But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. >> >> >> Given that a blueray movie is an array of bytes, is a text inappropriate? >> >> >> The actual diff presented just adjusts the limit by 8 bytes. >> Ok? >> Or must preserve pickles and can't do this w/o some other change? >> This isn't a more general problem? Or usually pickles >> are 32/64-compatible, but this is a most unusual case? >> >> >> Thanks, >> - Jay >> >> >> >> >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Wed, 3 Jun 2015 13:24:36 -0400 >> CC: m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> Why would you ever want a TEXT literal that is so large? >> By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface. >> A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. >> >> On Jun 3, 2015, at 12:36 PM, Jay wrote: >> >> I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. >> >> - Jay >> >> On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: >> >> Other than cross-compile from 32 to 64 bit what purpose does this serve? >> >> Sent from my iPad >> >> On Jun 2, 2015, at 9:58 PM, Jay K wrote: >> >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 >> index fa72589..37bf238 100644 >> --- a/m3-libs/m3core/src/text/TextLiteral.i3 >> +++ b/m3-libs/m3core/src/text/TextLiteral.i3 >> @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; >> IMPORT RTHooks, TextClass; >> >> CONST >> - (* DIV BITSIZE should not be here! *) >> +(* When cm3 uses TInt and when pickle special cases this, it should be approx: >> + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) >> +(* This fails for 32bit host and 64bit target: >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) >> +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) >> + dependent. *) >> + (* DIV BITSIZE should not be here -- compiler limitation due to >> + non-use of TInt. *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; >> (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) >> (* Do not adjust this for INTEGER size. It makes T have different >> fingerprints on 32- and 64-bit machines, which undermines pickling/ >> >> >> Yes I know TEXT pickles will be broken. >> I propose that unpickle should special case a few values here. >> Or, isn't the brand supposed to help? >> Or, can we encode this in a better way, w/o an upper bound? >> I know if you put in 0..0 or 0..-1 you incur range violations at runtime. >> Which makes me wonder -- are TEXTs unsafe? >> >> Should we support a syntax something like: >> >> cnt : INTEGER; >> buf : ARRAY [0..cnt - 1] OF Byte; >> ? >> >> Thanks, >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 4 18:35:07 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 11:35:07 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55705A41.7070503@elstel.org> References: <55705A41.7070503@elstel.org> Message-ID: <55707E3B.7020601@lcwb.coop> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: > Am 03.06.15 um 03:58 schrieb Jay K: >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> > > Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. > To me this is just another argument for a language specified value range of INTEGER. > It makes certain things easier. > > My suggestion was: > TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) > while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) > BITS 64 FOR INTEGER would only work on x64 systems. > What you want is already pretty much here. Just declare your desired type yourself: TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; It will always have this range and occupy 32 bits, regardless of the native word size. Pickles will always save and reread it with exactly this range, even between machines of different word size. The only thing that will differ from 32- to 64-bit machine is the size of intermediate results in expressions. If you are already _not_ being careful about avoiding overflows, there are probably cases where the results of silent overflows would differ with machine word size. Not sure what you would really want here. The unsigned arithmetic operations in Word are explicitly defined to silently do binary twos-complement wrap-around when overflows happen. The language does not define this for the builtin signed operators. In code where I care about overflows, I never assume this. It is probably machine-dependent. Or, you can import it: Int32.T. is already declared, in m3core, with this range. However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is almost never what you really want. The only place the BITS specification makes any difference is when you declare a record or object field of this type. In all other situations, BITS 32 FOR .. has _no effect_. The only purpose of BITS is so you can force the layout of a record. In that case, since the range needs exactly 32 bits anyway, even if you omit the BITS, a field of this type will still always occupy 32 bits. But, with the BITS specification, the compiler is not allowed to insert alignment padding, (if it were, you could not fully specify the layout) and if the field comes at a place that is not 32-bit aligned, you will either get a compile-time error or maybe a compiler will accept it and handle unaligned field access (it is not required to--ours does not), neither of which is probably what you want. Without BITS, the compiler will pad as needed. With BITS, you will have to insert explicit padding for alignment yourself as needed, which can be different for every field of this type of every record/object. And later adding/removing/changing any prior field can upset it and require you to rework the padding. Moreover, I am sure there are simple cases where the required padding differs between 32-bit and 64-bit machines. Having been bitten by this more than once, I now only put BITS..FOR as the anonymous type of a specific field, never in a named type to be used in multiple places. (Actually, BITS does the same thing when used as the type of an array element, but cases where this matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array fits in a single machine word. I do have a couple of such cases.) > The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value > range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for > any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). > > Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in > here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In > a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same > tends to be much slower). > > Regards, > Elmar > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Thu Jun 4 19:43:07 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Thu, 04 Jun 2015 19:43:07 +0200 Subject: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? In-Reply-To: <55707E3B.7020601@lcwb.coop> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> Message-ID: <55708E2B.4040607@elstel.org> Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem Am 04.06.15 um 18:35 schrieb Rodney M. Bates: > > > On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: >> Am 03.06.15 um 03:58 schrieb Jay K: >>> We cannot cross from 32bit host to 64bit target. >>> >>> >>> Ok to commit this? >>> >>> >> >> Compatibility between the 32bit and the 64bit version of the same >> program would be the minimum I expected from a pickle. >> To me this is just another argument for a language specified value >> range of INTEGER. >> It makes certain things easier. >> >> My suggestion was: >> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS >> 32 FOR INTEGER for 32 and 64bit targets) >> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I >> believe there is no reason to forbid it for INTEGER) >> BITS 64 FOR INTEGER would only work on x64 systems. >> > > What you want is already pretty much here. Just declare your desired > type yourself: > > TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; This should be good news. However my suggestion was to make Int32.T the default as this will be somewhat faster for the average application which is data intensive (just think of image processing) which will never be rewritten to use Int32.T instead of INT. All of our current and old programs being prepared to run on a 32bit as well as a 64bit target should compile and run without any problem. There is one single exception where the compiler would throw an error: when using INTEGER for address arithmetics in unsafe modules (error is: can only LOOPHOLE between types of the same size, or some way similar as I have already tested this). There we would need the so called targetint/offset. If I understand things right the following two things would need to get implemented to allow a default int size of 32 for existing applications (be it the default or just an additional compiler option as the size of WIDECHAR already is): * automatic range adaption for packed ints (i.e. those with BITS .. FOR) * allowing non 32bit alignment of INTs in memory (as is already the case for WIDE[CHAR]s) > > It will always have this range and occupy 32 bits, regardless of the > native word size. Pickles > will always save and reread it with exactly this range, even between > machines of different > word size. So the target size of the pickle is already determined by the respective INT subrange, right? Something that should to my believe be well documented as one could easily like to extend an enum with 255 members to more or equal than 256 an then wonder why the new an the old file formats become incompatible. To circumvent such a problem we would likely need to allow BITS 8*anything FOR INTEGER and take the bitcount specified in BITS for in order to determine the target size of an object instead of semi-automatically readjusting it all the time. > > The only thing that will differ from 32- to 64-bit machine is the size > of intermediate results in > expressions. If you are already _not_ being careful about avoiding > overflows, there are probably > cases where the results of silent overflows would differ with machine > word size. Not sure what > you would really want here. The unsigned arithmetic operations in > Word are explicitly defined > to silently do binary twos-complement wrap-around when overflows > happen. The language does not > define this for the builtin signed operators. In code where I care > about overflows, I never assume > this. It is probably machine-dependent. Yes, I believe it would be good like this. No real 32bit INT with 32bit arithmetics but just restricting the range for Int32.T when it gets written into memory. > > Or, you can import it: Int32.T. is already declared, in m3core, with > this range. > > However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is > almost never what you really want. > The only place the BITS specification makes any difference is when you > declare a record or object > field of this type. In all other situations, BITS 32 FOR .. has _no > effect_. The only purpose of > BITS is so you can force the layout of a record. In that case, since > the range needs exactly 32 > bits anyway, even if you omit the BITS, a field of this type will > still always occupy 32 bits. Nonetheless it is currently not possible to declare a variable of a 32bit type globally: TYPE Pixel = BITS 32 FOR INTEGER; unluckily does not work as we can have no global variable of type Pixel. This is not only un-orthogonal but also a very practical problem; Suggesting that we call procedures with a parameter of type pixel we will never be able to change that type f.i. to a packed record with members red, green, blue alpha if there are places in our main program where we need to declare VAR pix1, pix2 : INTEGER just because Pixel is a packed type being refused as global variable. > > But, with the BITS specification, the compiler is not allowed to > insert alignment padding, (if it > were, you could not fully specify the layout) and if the field comes > at a place that is not 32-bit > aligned, you will either get a compile-time error or maybe a compiler > will accept it and handle > unaligned field access (it is not required to--ours does not), neither > of which is probably what > you want. Without BITS, the compiler will pad as needed. With BITS, > you will have to insert explicit > padding for alignment yourself as needed, which can be different for > every field of this type of every > record/object. And later adding/removing/changing any prior field can > upset it and require you to > rework the padding. Moreover, I am sure there are simple cases where > the required padding differs > between 32-bit and 64-bit machines. On the long term I would personally even advocate an ALIGN directive: f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END; It would be very handy to have this for tapping the full power of the vectorization units of current processors. Manually re-aligning heap allocated objects in an unsafe module is not a very satisfying alternative (no local/global/stack allocation, memory waste and thus a certain performance loss if there are many wholes). However this would possibly also make changes to our garbage collector necessary as it currently only guarantees word alignment. Finally it would also be possible to write ALIGN BITSIZE(TARGETINT) .... > > Having been bitten by this more than once, I now only put BITS..FOR as > the anonymous type of a specific > field, never in a named type to be used in multiple places. A less bitchy and more orthogonal Modula-3 compiler supporting integer globals and locals of non word size and alignment would be a real pleasure, do you consent Rodney? > > (Actually, BITS does the same thing when used as the type of an array > element, but cases where this > matters are rare. If the bit count evenly divides the word size, > there won't be any alignment padding > needed, and BITS won't matter. Otherwise, the compiler is likely to > refuse, unless the entire array > fits in a single machine word. I do have a couple of such cases.) > >> The value range - bitsize correlation problem could be solved easily >> by automatically extending and reducing the value >> range of an integer type to what its binary representation allows as >> long as no explicit range has ever been specified for >> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR >> [-32768..+32767]). >> >> Finally it needs to be said that the argument that target size >> arithmetics is always the fastest which was often used in >> here can be very wrong. It does not hold for the 64bit systems of >> today because the CPU is no more the bottleneck. In >> a fact now the memory hierarchy has become the new bottleneck (which >> means that using more memory for the same >> tends to be much slower). >> >> Regards, >> Elmar >> > From mika at async.caltech.edu Thu Jun 4 19:46:29 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 04 Jun 2015 10:46:29 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55707E3B.7020601@lcwb.coop> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> Message-ID: <20150604174629.603F21A2065@async.async.caltech.edu> >(Actually, BITS does the same thing when used as the type of an array element, but cases where this >matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >fits in a single machine word. I do have a couple of such cases.) For a lot of hardware-related problems, the following type is quite useful: ARRAY [..] OF BITS 1 FOR BOOLEAN (and works with our compilers, too, as far as I know.) Mika From rodney_bates at lcwb.coop Thu Jun 4 19:51:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 12:51:43 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: <5570902F.3030401@lcwb.coop> On 06/04/2015 11:50 AM, Antony Hosking wrote: > >> On Jun 4, 2015, at 11:45 AM, Jay K > wrote: >> >> How strongly preferred and why? > > TInt preferred because LONGINT is still a terrible hack, and I hate to see it pollute the system more than it already has. > What?s more, the backends break on a number of operations for it. Danger, Will Robinson! This could introduce truly show-stopping bootstrap problems, using LONGINT in the compiler, and in places essential to its ability to function at all. One little bug and we could end up with neither a chicken nor an egg. > I now regret ever having introduced it, and would like to back it out. We would have been better to specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF INTEGER, and then hidden manipulation of those offsets behind a clean interface, much as Tint does. I think the one use-case we had for it to allow interfacing directly to C functions that take offset_t arguments could have been finessed differently. > >> good naturedly: >> >> >> 1) to cause me pain so that might give up? >> 2) to take more time so I don't muck with other stuff? >> 3) for compatibility with older releases? >> 4) for easier extension in future? >> 5) to cause me pain so I whine more about wanting operator overloading? > > Perhaps 3 & 4. > >> I started this once and it going to be a pain. >> LONGINT is probably much easier. >> Maybe I have more patience now. > > :-) Patience accrues with age? > >> Extension in the future could be addition of INTEGER128 to the language. :) >> And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128 >> and convert frontend to use it. Hypothetically. Perhaps before that happens >> we'll have operator overloading. :) > > Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. > >> >> >> - Jay >> >> >> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- --- >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Thu, 4 Jun 2015 11:32:48 -0400 >> CC: rodney.m.bates at acm.org ; m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> Again, TInt preferred. >> >> Sent from my iPhone >> >> On Jun 4, 2015, at 11:08 AM, Jay K > wrote: >> >> When I proposed TInt I had actually forgotten about LONGINT! >> LONGINT is easier to code to (until/unless we get operator overloading...) >> >> TInt is easier to extend, and is portable to before the current release. >> I think LONGINT is adequate. >> I did extent TInt to 72 bits I think. >> >> - Jay >> >> >> > Date: Thu, 4 Jun 2015 09:37:06 -0500 >> > From:rodney_bates at lcwb.coop >> > To:jay.krell at cornell.edu ;hosking at purdue.edu >> > CC:m3devel at elegosoft.com >> > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> > >> > >> > >> > On 06/03/2015 06:14 PM, Jay K wrote: >> > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. >> > > An array of chars? Probably has same problem. >> > > >> > > >> > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? >> > > >> > > - Jay >> > > >> > >> > I'm OK with doing it, but I thought you had been talking about using TInt. >> > Would that be better? It would be more general. But LONGINT would be >> > faster at compile time, and less work, since the arithmetic operators >> > would not need to be changed to TInt function calls. There might >> > be quite a lot of those I guess the compiler would be of about equal >> > help in finding missed places to be changed, either way. >> > >> > I just checked, and LONGINT is in the release compiler, contrary to >> > my sense of relative history, so there would be no bootstrap barrier. >> > >> > Either would allow a 32-bit hosted compiler (cross- or native-) to handle >> > types whose byte-count approached 2^31, instead of just 2^23. With a >> > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, >> > whereas LONGINT would only go to 2^55. Do we really care? >> > >> > > >> > > >> > > >> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----- >> > -- >> > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> > > From:hosking at purdue.edu >> > > Date: Wed, 3 Jun 2015 14:07:02 -0400 >> > > CC:m3devel at elegosoft.com >> > > To:jay.krell at cornell.edu >> > > >> > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. >> > > >> > > >> > -- >> > Rodney Bates >> >rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From adacore at marino.st Thu Jun 4 19:57:23 2015 From: adacore at marino.st (John Marino) Date: Thu, 04 Jun 2015 19:57:23 +0200 Subject: [M3devel] rpath woes Message-ID: <55709183.4060804@marino.st> I'm been finished with my port update for more than 1 day, except for one tiny problem. The rpath was wrong after I shifted the installation to /usr/local/cm3. I have tried just about every combination of changes to files at m3-sys/cminstall/src/config-no-install and I can't get the rpath to change. It's set at "/usr/local/lib", I don't know where it comes from. I want it to be "/usr/local/cm3/lib:/usr/local/lib". At gnuld.common, there are weird definitions of "-Wl,-rpath," but altering them doesn't help (not even "-W1,-rpath,//$ORIGIN"/../lib" So I am at a loss where rpath is getting set. If I knew that, I could change it. It seems to ignore all this SYSTEM_LD stuff so it must not be set. Any hints or do I have to keep randomly changing anything like looks like it could be a linker flag? John From jay.krell at cornell.edu Thu Jun 4 20:01:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 18:01:05 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, <55706292.9060608@lcwb.coop>, , <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu>, , Message-ID: > Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. The combination of features I want hasn't been materialized. Something like: 1) operator overloading -- C++ 2) templates with type deduction (i.e. C++ function templates, not just class templates or generic modules) -- C++ 3) a small easy to understand language -- not C++, maybe Modula-3 4) a small easy to understand compiler (m3front I don't understand) -- nowhere 5) fast compilation -- Modula-3 6) optional safey -- Modula-3 7) multiple inheritance, at least of "interfaces" -- C++ 8) optionally stack or inline allocation of "classes"/"objects" -- C++ 9) clear choice of build system -- Modula-3 perhaps 10) locals with destructors -- C++ 11) clear portable choice for remoting/RPC -- Modula-3 perhaps 12) needs backend work -- Modula-3 perhaps 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta 14) ?safety w/o garbage collection? -- rust?? 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. Not Python/Perl/Ruby/Erlang/sh/Tcl. 16) static compilation to native code, maybe -- C++ and Modula-3! Go? Net.Native? 17) non-virtual member functions -- not C or Modula-3 18) virtual member functions -- C++, Modula-3 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ was late here. 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 and C are decades late. 22) good string library 23) good regex library Basically, for now, I want static compilation to native code, fast compilation, optional safety. That combination is rare. Rust is unusual in safety w/o gc. Extended static lifetime analysis/verification... - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 12:50:02 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu On Jun 4, 2015, at 11:45 AM, Jay K wrote: How strongly preferred and why? TInt preferred because LONGINT is still a terrible hack, and I hate to see it pollute the system more than it already has. What?s more, the backends break on a number of operations for it. I now regret ever having introduced it, and would like to back it out. We would have been better to specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF INTEGER, and then hidden manipulation of those offsets behind a clean interface, much as Tint does. I think the one use-case we had for it to allow interfacing directly to C functions that take offset_t arguments could have been finessed differently. good naturedly: 1) to cause me pain so that might give up? 2) to take more time so I don't muck with other stuff? 3) for compatibility with older releases? 4) for easier extension in future? 5) to cause me pain so I whine more about wanting operator overloading? Perhaps 3 & 4. I started this once and it going to be a pain. LONGINT is probably much easier. Maybe I have more patience now. :-) Patience accrues with age? Extension in the future could be addition of INTEGER128 to the language. :)And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128and convert frontend to use it. Hypothetically. Perhaps before that happenswe'll have operator overloading. :) Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 11:32:48 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu Again, TInt preferred. Sent from my iPhone On Jun 4, 2015, at 11:08 AM, Jay K wrote: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 20:05:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 18:05:00 +0000 Subject: [M3devel] rpath woes In-Reply-To: <55709183.4060804@marino.st> References: <55709183.4060804@marino.st> Message-ID: My understanding: The modula-3 libraries, e.g. m3core,are found from the rpath I put in, like $ORIGIN:$ORIGIN/../lib Other libraries like libc.so are found by full path, that is already correct at build time? Do users want to possibly have a private /usr/local/lib/libc.so?I hadn't considered that. Or build is "staged" and the full paths aren't correct at build time?I think I hadn't considered that either. You are on the right track, looking in this area. Did you notice the "portable rpath" thingy that make-dist.py changes? - Jay > Date: Thu, 4 Jun 2015 19:57:23 +0200 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] rpath woes > > I'm been finished with my port update for more than 1 day, except for > one tiny problem. The rpath was wrong after I shifted the installation > to /usr/local/cm3. > > I have tried just about every combination of changes to files at > m3-sys/cminstall/src/config-no-install and I can't get the rpath to change. > > It's set at "/usr/local/lib", I don't know where it comes from. I want > it to be "/usr/local/cm3/lib:/usr/local/lib". > > At gnuld.common, there are weird definitions of "-Wl,-rpath," but > altering them doesn't help (not even "-W1,-rpath,//$ORIGIN"/../lib" > > So I am at a loss where rpath is getting set. If I knew that, I could > change it. It seems to ignore all this SYSTEM_LD stuff so it must not > be set. > > Any hints or do I have to keep randomly changing anything like looks > like it could be a linker flag? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Thu Jun 4 20:10:07 2015 From: adacore at marino.st (John Marino) Date: Thu, 04 Jun 2015 20:10:07 +0200 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st> Message-ID: <5570947F.2080600@marino.st> On 6/4/2015 20:05, Jay K wrote: > My understanding: > The modula-3 libraries, e.g. m3core, > are found from the rpath I put in, like > $ORIGIN:$ORIGIN/../lib > Other libraries like libc.so are found by full path, that is already > correct at build time? > Do users want to possibly have a private /usr/local/lib/libc.so? > I hadn't considered that. > Or build is "staged" and the full paths aren't correct at build time? > I think I hadn't considered that either. > You are on the right track, looking in this area. > Did you notice the "portable rpath" thingy that make-dist.py changes? I believe make-dist.py is hard-coding portable rpath which means it's hardcoding $ORIGIN I think, but it's not working. The origin flag is set, but $ORIGIN itself is not. I think this is a bug. We prefer to use hardcoded rpaths in ports, but I will settle for $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not NetBSD (not sure about OpenBSD, I'd guess it does though). The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't work either. John From hendrik at topoi.pooq.com Thu Jun 4 20:24:44 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 4 Jun 2015 14:24:44 -0400 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st> Message-ID: <20150604182444.GB8703@topoi.pooq.com> On Thu, Jun 04, 2015 at 06:05:00PM +0000, Jay K wrote: > Do users want to possibly have a private /usr/local/lib/libc.so?I > hadn't considered that. The musl users might. It's another C runtime system which may have some standards-compliance and efficiency advantages. see http://www.musl-libc.org/ -- hemdrik From lists at darko.org Thu Jun 4 20:32:17 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 4 Jun 2015 11:32:17 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: You really should try writing your own language, it's really not that hard. M3 will never give you satisfaction. You've already implemented a C back end and no doubt have an idea of how a compiler works. As for language features, I think we should be looking for features to remove. On Thu, Jun 4, 2015 at 11:01 AM, Jay K wrote: > > Operator overloading is anathema to the Modula-3 design principles. > Use C++ if that is what you want. > > > The combination of features I want hasn't been materialized. > Something like: > > 1) operator overloading -- C++ > 2) templates with type deduction (i.e. C++ function templates, not just > class templates or generic modules) -- C++ > 3) a small easy to understand language -- not C++, maybe Modula-3 > 4) a small easy to understand compiler (m3front I don't understand) -- > nowhere > 5) fast compilation -- Modula-3 > 6) optional safey -- Modula-3 > 7) multiple inheritance, at least of "interfaces" -- C++ > 8) optionally stack or inline allocation of "classes"/"objects" -- C++ > 9) clear choice of build system -- Modula-3 perhaps > 10) locals with destructors -- C++ > 11) clear portable choice for remoting/RPC -- Modula-3 perhaps > 12) needs backend work -- Modula-3 perhaps > 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta > 14) ?safety w/o garbage collection? -- rust?? > 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. > Not Python/Perl/Ruby/Erlang/sh/Tcl. > 16) static compilation to native code, maybe -- C++ and Modula-3! Go? > Net.Native? > 17) non-virtual member functions -- not C or Modula-3 > 18) virtual member functions -- C++, Modula-3 > 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) > 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ > was late here. > 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 > and C are decades late. > 22) good string library > 23) good regex library > > > Basically, for now, I want static compilation to native code, fast > compilation, optional safety. > That combination is rare. > > > Rust is unusual in safety w/o gc. Extended static lifetime > analysis/verification... > > > - Jay > > > > ------------------------------ > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > From: hosking at purdue.edu > Date: Thu, 4 Jun 2015 12:50:02 -0400 > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > To: jay.krell at cornell.edu > > > On Jun 4, 2015, at 11:45 AM, Jay K wrote: > > How strongly preferred and why? > > > TInt preferred because LONGINT is still a terrible hack, and I hate to see > it pollute the system more than it already has. What?s more, the backends > break on a number of operations for it. I now regret ever having > introduced it, and would like to back it out. We would have been better to > specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF > INTEGER, and then hidden manipulation of those offsets behind a clean > interface, much as Tint does. I think the one use-case we had for it to > allow interfacing directly to C functions that take offset_t arguments > could have been finessed differently. > > good naturedly: > > > 1) to cause me pain so that might give up? > 2) to take more time so I don't muck with other stuff? > 3) for compatibility with older releases? > 4) for easier extension in future? > 5) to cause me pain so I whine more about wanting operator overloading? > > > Perhaps 3 & 4. > > I started this once and it going to be a pain. > LONGINT is probably much easier. > Maybe I have more patience now. > > > :-) Patience accrues with age? > > Extension in the future could be addition of INTEGER128 to the language. :) > And the 128 bit targets will initially have 64bit LONGINT limits, until we > add INTEGER128 > and convert frontend to use it. Hypothetically. Perhaps before that happens > we'll have operator overloading. :) > > > Operator overloading is anathema to the Modula-3 design principles. Use > C++ if that is what you want. > > > > - Jay > > > ------------------------------ > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > From: hosking at purdue.edu > Date: Thu, 4 Jun 2015 11:32:48 -0400 > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > To: jay.krell at cornell.edu > > Again, TInt preferred. > > Sent from my iPhone > > On Jun 4, 2015, at 11:08 AM, Jay K wrote: > > When I proposed TInt I had actually forgotten about LONGINT! > LONGINT is easier to code to (until/unless we get operator overloading...) > > TInt is easier to extend, and is portable to before the current release. > I think LONGINT is adequate. > I did extent TInt to 72 bits I think. > > - Jay > > > > Date: Thu, 4 Jun 2015 09:37:06 -0500 > > From: rodney_bates at lcwb.coop > > To: jay.krell at cornell.edu; hosking at purdue.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > > > > > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > > sorry, I missed that it is literals...so I can convert a blueray movie > into a source file containing the data all in a text? Far fetched. > > > An array of chars? Probably has same problem. > > > > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and > field offsets? > > > > > > - Jay > > > > > > > I'm OK with doing it, but I thought you had been talking about using > TInt. > > Would that be better? It would be more general. But LONGINT would be > > faster at compile time, and less work, since the arithmetic operators > > would not need to be changed to TInt function calls. There might > > be quite a lot of those I guess the compiler would be of about equal > > help in finding missed places to be changed, either way. > > > > I just checked, and LONGINT is in the release compiler, contrary to > > my sense of relative history, so there would be no bootstrap barrier. > > > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > > types whose byte-count approached 2^31, instead of just 2^23. With a > > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > -- > > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > > > From: hosking at purdue.edu > > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > > CC: m3devel at elegosoft.com > > > To: jay.krell at cornell.edu > > > > > > It?s only TEXT /literals/ that are limited here. As in, what appears > in a source program. > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 5 00:24:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 17:24:26 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <20150604174629.603F21A2065@async.async.caltech.edu> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> <20150604174629.603F21A2065@async.async.caltech.edu> Message-ID: <5570D01A.5000909@lcwb.coop> On 06/04/2015 12:46 PM, mika at async.caltech.edu wrote: >> (Actually, BITS does the same thing when used as the type of an array element, but cases where this >> matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >> needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >> fits in a single machine word. I do have a couple of such cases.) > > For a lot of hardware-related problems, the following type is quite useful: > > ARRAY [..] OF BITS 1 FOR BOOLEAN > > (and works with our compilers, too, as far as I know.) > Yeah, I forgot about that group of cases. Without a BITS spec, the only sizes the compiler will choose are 64, 32, 16, or 8. If you want 4, 2, or 1, or something else that doesn't divide 32, you have to use BITS. But they will work fine. > Mika > -- Rodney Bates rodney.m.bates at acm.org From adacore at marino.st Fri Jun 5 00:28:36 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 00:28:36 +0200 Subject: [M3devel] rpath woes In-Reply-To: <5570947F.2080600@marino.st> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> Message-ID: <5570D114.40104@marino.st> On 6/4/2015 20:10, John Marino wrote: > On 6/4/2015 20:05, Jay K wrote: >> My understanding: >> The modula-3 libraries, e.g. m3core, >> are found from the rpath I put in, like > >> $ORIGIN:$ORIGIN/../lib >> Other libraries like libc.so are found by full path, that is already >> correct at build time? >> Do users want to possibly have a private /usr/local/lib/libc.so? >> I hadn't considered that. >> Or build is "staged" and the full paths aren't correct at build time? >> I think I hadn't considered that either. >> You are on the right track, looking in this area. >> Did you notice the "portable rpath" thingy that make-dist.py changes? > > I believe make-dist.py is hard-coding portable rpath which means it's > hardcoding $ORIGIN I think, but it's not working. > > The origin flag is set, but $ORIGIN itself is not. I think this is a bug. > > We prefer to use hardcoded rpaths in ports, but I will settle for > $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not > NetBSD (not sure about OpenBSD, I'd guess it does though). > > The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't > work either. Okay, I have this solved. The problem is FREEBSD.common is obsolete (origin linker flags were wrong) I basically copied LINUX.common contents to it and now everything works as expected. I've updated the port here: http://www.freshports.org/lang/modula3/ The patch for FREEBSD.common is here: https://svnweb.freebsd.org/ports/head/lang/modula3/files/ I think all references to FREEBSD4 could be removed (as I did in the patch). FreeBSD4 was EOL in Jan 2007. So I'm basically done until its time to change the port to use the C backend (and/or bootstrap it to DragonFly) John From jay.krell at cornell.edu Fri Jun 5 06:16:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 5 Jun 2015 04:16:24 +0000 Subject: [M3devel] rpath woes In-Reply-To: <5570D114.40104@marino.st> References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>,<5570D114.40104@marino.st> Message-ID: The FreeBSD4 users were surprisingly vocal surprisingly recently. So I put some work into it. I agree it is an old system. Linux.common and FreeBSD.common look very similar but I agree different and if it is working for you, good, I can ignore it? I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? Was this right for FreeBSD 5 or 6 or 7 or 8? Dragonfly should be fairly easy...though it is a bit more work than is ideal. The main thing is we still have lists of targets. Look in m3-sys/m3middle/src/Target.* Soon hopefully the jmpbuf there will go away. That will help. Areas to port are easy to find with grep/findstr: C:\dev2\cm3\m3-libs\libm3>findstr /m /s /i OPENBSD * src\os\POSIX\m3makefile src\os\POSIX\m3makefile-old src\uid\POSIX\getMID.c src\uid\POSIX\MachineIDPosixC.c tests\random\src\m3makefile C:\dev2\cm3\m3-libs\m3core>findstr /m /s /i OPENBSD * src\atomic\m3makefile src\C\Common\Cstring.i3 src\Csupport\Common\lround-readme.txt src\Csupport\Common\s_llroundl.c src\Csupport\Common\s_lround.c src\Csupport\Common\s_lroundl.c src\Csupport\m3makefile-old src\float\m3makefile src\float\m3makefile-old src\m3core.h src\runtime\POSIX\RTSignalC.c src\thread\POSIX\ThreadPosixC.c src\thread\PTHREAD\m3makefile src\thread\PTHREAD\ThreadOpenBSD.c src\thread\PTHREAD\ThreadPThreadC.c src\time\POSIX\m3makefile.old src\unix\Common\Ustat.i3 src\unix\m3makefile src\unix\m3makefile-old C:\dev2\cm3\m3-sys\m3middle>findstr /m /s /i OPENBSD * src\Target.i3 src\Target.i3-old src\Target.m3 src\Target.m3-old This looks scarier than it is. Many cases are just to add to a list of platforms you are close enough to. Some cases I hope to eliminate -- cooperative suspend will cut out a swath for example. Thanks, - Jay > Date: Fri, 5 Jun 2015 00:28:36 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > > On 6/4/2015 20:10, John Marino wrote: > > On 6/4/2015 20:05, Jay K wrote: > >> My understanding: > >> The modula-3 libraries, e.g. m3core, > >> are found from the rpath I put in, like > > > >> $ORIGIN:$ORIGIN/../lib > >> Other libraries like libc.so are found by full path, that is already > >> correct at build time? > >> Do users want to possibly have a private /usr/local/lib/libc.so? > >> I hadn't considered that. > >> Or build is "staged" and the full paths aren't correct at build time? > >> I think I hadn't considered that either. > >> You are on the right track, looking in this area. > >> Did you notice the "portable rpath" thingy that make-dist.py changes? > > > > I believe make-dist.py is hard-coding portable rpath which means it's > > hardcoding $ORIGIN I think, but it's not working. > > > > The origin flag is set, but $ORIGIN itself is not. I think this is a bug. > > > > We prefer to use hardcoded rpaths in ports, but I will settle for > > $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not > > NetBSD (not sure about OpenBSD, I'd guess it does though). > > > > The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't > > work either. > > Okay, I have this solved. > The problem is FREEBSD.common is obsolete (origin linker flags were > wrong) I basically copied LINUX.common contents to it and now > everything works as expected. > > I've updated the port here: > http://www.freshports.org/lang/modula3/ > > The patch for FREEBSD.common is here: > https://svnweb.freebsd.org/ports/head/lang/modula3/files/ > > I think all references to FREEBSD4 could be removed (as I did in the > patch). FreeBSD4 was EOL in Jan 2007. > > So I'm basically done until its time to change the port to use the C > backend (and/or bootstrap it to DragonFly) > > John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jun 5 07:50:33 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 07:50:33 +0200 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>, <5570D114.40104@marino.st> Message-ID: <557138A9.3000605@marino.st> On 6/5/2015 06:16, Jay K wrote: > The FreeBSD4 users were surprisingly vocal surprisingly recently. > So I put some work into it. > I agree it is an old system. It's completely unsupported (for 7 years now) and they don't need the latest M3 right? Old release still work fine, that should be enough justification to change this on trunk. > Linux.common and FreeBSD.common look very similar but I agree different > and if it is working for you, good, I can ignore it? > I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? > Was this right for FreeBSD 5 or 6 or 7 or 8? yes. I am using the latest binutils (thus latest ld) with M3. The base ld is ancient and hand maintained (a GPLv3 issue) so I don't know, Actually, the linux way is opposite: It explicitly defines SYSTEM_LD rather than use flags for SYSTEM_CC. So rather than flags, we are talking about the approach. Really nobody with a supported FreeBSD release needs to build from scratch because it exists in ports. If anybody absolutely wants to, just say "emulates what the port is doing". This is actually easier said than done because in addition to using the latest binutils, there are a number of inline substitutions that are done, see post-patch target here, starting line 78: https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup line 85 is where newer gcc and newer as is set for m3. John From adacore at marino.st Fri Jun 5 08:10:08 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 08:10:08 +0200 Subject: [M3devel] install issues (stripping, permissions) Message-ID: <55713D40.6080305@marino.st> This is old news but I never reported it. See the install target on the port's makefile (starting line 100): https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup I have to due numerous changes to the installed files to make it conform to ports standards. The issues involved stripping and permissions. For ports, executables are installed stripped unless explicitly wanted. Usually gnu makefiles support this with "install" and "install-strip" targets. This can be done after the fact, as I have done, with the ${STRIP_CMD} command, which does nothing when stripping is not desired. The other issue is that there were numerous files in pkg/ that were executable and shouldn't have been. I used find/chmod to remove exec perms for all except "cm3" file. I did this 1.5 years ago, I'm now wondering how right it was. The current list: pkg/m3core/src/C/Common/Csetjmp.i3 pkg/m3back/src/M3C.i3 pkg/m3staloneback/AMD64_FREEBSD/m3back pkg/cm3/AMD64_FREEBSD/cm3 pkg/libdump/AMD64_FREEBSD/libdump pkg/cmpfp/AMD64_FREEBSD/cmpfp pkg/formsview/AMD64_FREEBSD/formsview pkg/vorun/AMD64_FREEBSD/vorun pkg/pkl-fonts/AMD64_FREEBSD/PklFonts pkg/hack/AMD64_FREEBSD/dummy pkg/test/AMD64_FREEBSD/test Should there be executables in pkg/ ? Is the chmod command then wrong? I had to strips these as well to pass QA tests. John From wagner at elegosoft.com Fri Jun 5 09:26:18 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 5 Jun 2015 09:26:18 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55713D40.6080305@marino.st> References: <55713D40.6080305@marino.st> Message-ID: <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> On Fri, 05 Jun 2015 08:10:08 +0200 John Marino wrote: > The other issue is that there were numerous files in pkg/ that were > executable and shouldn't have been. I used find/chmod to remove exec > perms for all except "cm3" file. I did this 1.5 years ago, I'm now > wondering how right it was. The current list: > > pkg/m3core/src/C/Common/Csetjmp.i3 > pkg/m3back/src/M3C.i3 > pkg/m3staloneback/AMD64_FREEBSD/m3back > pkg/cm3/AMD64_FREEBSD/cm3 > pkg/libdump/AMD64_FREEBSD/libdump > pkg/cmpfp/AMD64_FREEBSD/cmpfp > pkg/formsview/AMD64_FREEBSD/formsview > pkg/vorun/AMD64_FREEBSD/vorun > pkg/pkl-fonts/AMD64_FREEBSD/PklFonts > pkg/hack/AMD64_FREEBSD/dummy > pkg/test/AMD64_FREEBSD/test > > Should there be executables in pkg/ ? Is the chmod command then wrong? > I had to strips these as well to pass QA tests. Except for the i3-files these are all programs and should be executable. The builder ships programs to the package target directory, and Programs with a capital P to the bin directory. As I said, M3 packages don't fit well into OS packages. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From estellnb at elstel.org Fri Jun 5 10:21:31 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:21:31 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <7427033D-A18B-4F81-A823-229F91313F16@purdue.edu> References: <55705A41.7070503@elstel.org> <7427033D-A18B-4F81-A823-229F91313F16@purdue.edu> Message-ID: <55715C0B.8090207@elstel.org> Antony, you do not seem to understand what I have proposed. Please have a look at my latest mail about Re: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? Am 04.06.15 um 19:14 schrieb Antony Hosking: > >> On Jun 4, 2015, at 10:01 AM, Elmar Stellnberger > > wrote: >> >> Am 03.06.15 um 03:58 schrieb Jay K: >>> We cannot cross from 32bit host to 64bit target. >>> >>> >>> Ok to commit this? >>> >>> >> >> Compatibility between the 32bit and the 64bit version of the same >> program would be the minimum I expected from a pickle. >> To me this is just another argument for a language specified value >> range of INTEGER. >> It makes certain things easier. >> >> My suggestion was: >> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS >> 32 FOR INTEGER for 32 and 64bit targets) >> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I >> believe there is no reason to forbid it for INTEGER) >> BITS 64 FOR INTEGER would only work on x64 systems. > > You are still misunderstanding the purpose of BITS FOR. It is used to > express packing requirements for use of those types *inside* > structured values (such as RECORD or ARRAY) to enforce a particular > bit-packing and to avoid alignment contraints. What it means is that > bit-wise operations must be used to load/store the value in memory in > the case that aligned operations do not suffice. > >> >> The value range - bitsize correlation problem could be solved easily >> by automatically extending and reducing the value >> range of an integer type to what its binary representation allows as >> long as no explicit range has ever been specified for >> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR >> [-32768..+32767]). > > A value of type [-32768..+32767] *never* occupies more than 16 bits in > memory! > >> Finally it needs to be said that the argument that target size >> arithmetics is always the fastest which was often used in >> here can be very wrong. It does not hold for the 64bit systems of >> today because the CPU is no more the bottleneck. In >> a fact now the memory hierarchy has become the new bottleneck (which >> means that using more memory for the same >> tends to be much slower). > > Operating on 16-bit values using natural word-size operations (32 bit > or 64 bit) is never more expensive. And the memory is only ever > 16-bit. So your performance argument does not stand. > > I hope this clarifies your understanding of the Modula-3 type system. > >> >> Regards, >> Elmar >> >>> Can't do this. Non-open arrays must have static constant bounds, >>> which the above does not. Open arrays have extra dope and are accessed >>> through two extra pointers, which we really wouldn't want for text >>> literals. >>> >>> TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse >>> that anything >>> else in UNSAFE code. The interface is UNSAFE, which means safe code >>> can't IMPORT >>> it, and thus can't see the declaration. Not very likely that >>> anybody other than >>> the runtime, compiler-generated code, and debugger (written in C >>> anyway) would >>> want to mess with it. >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Fri Jun 5 10:24:32 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:24:32 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: <55715CC0.401@elstel.org> Am 04.06.15 um 20:32 schrieb Darko Volaric: > You really should try writing your own language, it's really not that > hard. Darko, you are dreaming. Writing a small demo compiler may be one weeks job. However writing something that implements all that your hart desires will be very hard to do if at all doable for a single person. > M3 will never give you satisfaction. You've already implemented a C > back end and no doubt have an idea of how a compiler works. > > As for language features, I think we should be looking for features to > remove. > Jay, with a little more time I would have been at your side in a first place! A couple of years ago I had some own plans for a M3/Pascal like language with similar or even more venturous properties. 1) operator overloading -- C++ 2) templates with type deduction (i.e. C++ function templates, not just class templates or generic modules) -- C++ 3) a small easy to understand language -- not C++, maybe Modula-3 4) a small easy to understand compiler (m3front I don't understand) -- nowhere 5) fast compilation -- Modula-3 6) optional safey -- Modula-3 7) multiple inheritance, at least of "interfaces" -- C++ 8) optionally stack or inline allocation of "classes"/"objects" -- C++ 9) clear choice of build system -- Modula-3 perhaps 10) locals with destructors -- C++ 11) clear portable choice for remoting/RPC -- Modula-3 perhaps 12) needs backend work -- Modula-3 perhaps 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta 14) ?safety w/o garbage collection? -- rust?? 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. Not Python/Perl/Ruby/Erlang/sh/Tcl. 16) static compilation to native code, maybe -- C++ and Modula-3! Go? Net.Native? 17) non-virtual member functions -- not C or Modula-3 18) virtual member functions -- C++, Modula-3 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ was late here. 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 and C are decades late. 22) good string library 23) good regex library Nonetheless I wanna see what can be done with Modula-3 in its current state. - and I believe it already offers some interesting properties compared to C like f.i. range checking and better type safety which could make a difference for security relevant projects like f.i. a web browser. The only thing that would be necessary for such a project was a *** well tested and ready-to-use shipped *** GUI toolkit ... From estellnb at elstel.org Fri Jun 5 10:33:54 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:33:54 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55715CC0.401@elstel.org> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> <55715CC0.401@elstel.org> Message-ID: <55715EF2.7020500@elstel.org> > 17) non-virtual member functions -- not C or Modula-3 That already works for Modula-3. Simply put the method name with the same signature another time under "methods" instead of overrides (However you would have to do that in each subclass - and there is likely no runtime optimization for it; i.e. it may effectively just declare another virtual method with the same name.). From estellnb at elstel.org Fri Jun 5 10:29:24 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:29:24 +0200 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <556BD24D.7010902@lcwb.coop> References: <20150601002944.GA10751@topoi.pooq.com> <556BD24D.7010902@lcwb.coop> Message-ID: <55715DE4.3000304@elstel.org> Am 01.06.15 um 05:32 schrieb Rodney M. Bates: > There is a QT binding in cm3/m3-ui/qt. I have no experience > with it. > Dear Hendrik, I am also very interested in a good GUI environment for Modula-3. It would be very nice if you could share your experience with the current Qt bindings if you should resolve to give them a test. Once I had started to port the program at http://www.elstel.org/coan/ to Modula-3. That time there were no Qt-bindings and I had started to program my own bindings for M3. As far as I can remember they use virtual methods instead of signals or slots. I am very likely to continue my work on it at least if the current Qt toolkit should not be as good as desired. By the end of this summer I would suppose CoAn to be available under an OSSy license and being ported to Modula-3. Elmar > On 05/31/2015 07:29 PM, Hendrik Boom wrote: >> What user interface libraries are available for Modula 3? >> >> I know there's Trestle. >> >> But is there something like GTK or QT or somethng I can use like cairo >> to draw really pretty text? Anything that supports lots of Unicode? >> I don't mind havein to do my own character placement based on font >> metrics of some sort; not now, bit later, I may have some really >> weird layout constraints. >> >> In case you're wodering about the application: >> >> I'm doing preliminary planning for something like a text editor with >> several windows, one with the raw text, and another with a continuous >> (as continuous as I have cpu cycles for) display of the results of >> rather >> complex calculations on that text (such as proof checking or >> described graphics). >> >> Modula 3 isn't the only candidate for a programming language, just the >> leading candidate for historical and bootstrapping reasons. The others >> at the moment are OCAML and some kind of statically typed Scheme. >> >> And of course, I may decide that theo whole projecct is infeasible. >> >> -- hendrik >> >> > From adacore at marino.st Fri Jun 5 11:14:50 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 11:14:50 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> Message-ID: <5571688A.4060404@marino.st> On 6/5/2015 09:26, Olaf Wagner wrote: > On Fri, 05 Jun 2015 08:10:08 +0200 > John Marino wrote: > >> The other issue is that there were numerous files in pkg/ that were >> executable and shouldn't have been. I used find/chmod to remove exec >> perms for all except "cm3" file. I did this 1.5 years ago, I'm now >> wondering how right it was. The current list: >> >> pkg/m3core/src/C/Common/Csetjmp.i3 >> pkg/m3back/src/M3C.i3 >> pkg/m3staloneback/AMD64_FREEBSD/m3back >> pkg/cm3/AMD64_FREEBSD/cm3 >> pkg/libdump/AMD64_FREEBSD/libdump >> pkg/cmpfp/AMD64_FREEBSD/cmpfp >> pkg/formsview/AMD64_FREEBSD/formsview >> pkg/vorun/AMD64_FREEBSD/vorun >> pkg/pkl-fonts/AMD64_FREEBSD/PklFonts >> pkg/hack/AMD64_FREEBSD/dummy >> pkg/test/AMD64_FREEBSD/test >> >> Should there be executables in pkg/ ? Is the chmod command then wrong? >> I had to strips these as well to pass QA tests. > > Except for the i3-files these are all programs and should be executable. Okay, when I fixed this internally, I discovered something: m3back, libdump, cmpfp, formsview, vorun, dummy, test all link to m3 libraries but the rpath of $ORIGIN:$ORIGIN/../lib isn't enough. All of these programs need an additional rpath to $ORIGIN/../../../lib for the linker to find the libs. These all need to be fixed in repository I suspect. John From lists at darko.org Fri Jun 5 13:07:20 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 5 Jun 2015 04:07:20 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55715CC0.401@elstel.org> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> <55715CC0.401@elstel.org> Message-ID: I guess it depends on your level of skills, but source to source compilers aren't that challenging. The more important point is that exercise might focus Jay's mind on the the complexity and desirability of those features. It's all fine talking about them in the abstract, it's completely different when you get down to brass tacks. On Fri, Jun 5, 2015 at 1:24 AM, Elmar Stellnberger wrote: > Am 04.06.15 um 20:32 schrieb Darko Volaric: > >> You really should try writing your own language, it's really not that >> hard. >> > Darko, you are dreaming. Writing a small demo compiler may be one weeks > job. > However writing something that implements all that your hart desires will > be > very hard to do if at all doable for a single person. > > M3 will never give you satisfaction. You've already implemented a C back >> end and no doubt have an idea of how a compiler works. >> >> As for language features, I think we should be looking for features to >> remove. >> >> Jay, with a little more time I would have been at your side in a first > place! > A couple of years ago I had some own plans for a M3/Pascal like language > with similar or even more venturous properties. > > 1) operator overloading -- C++ > 2) templates with type deduction (i.e. C++ function templates, not just > class templates or generic modules) -- C++ > 3) a small easy to understand language -- not C++, maybe Modula-3 > 4) a small easy to understand compiler (m3front I don't understand) -- > nowhere > 5) fast compilation -- Modula-3 > 6) optional safey -- Modula-3 > 7) multiple inheritance, at least of "interfaces" -- C++ > 8) optionally stack or inline allocation of "classes"/"objects" -- C++ > 9) clear choice of build system -- Modula-3 perhaps > 10) locals with destructors -- C++ > 11) clear portable choice for remoting/RPC -- Modula-3 perhaps > 12) needs backend work -- Modula-3 perhaps > 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta > 14) ?safety w/o garbage collection? -- rust?? > 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. > Not Python/Perl/Ruby/Erlang/sh/Tcl. > 16) static compilation to native code, maybe -- C++ and Modula-3! Go? > Net.Native? > 17) non-virtual member functions -- not C or Modula-3 > 18) virtual member functions -- C++, Modula-3 > 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) > 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ > was late here. > 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 > and C are decades late. > 22) good string library > 23) good regex library > > Nonetheless I wanna see what can be done with Modula-3 in its current > state. - > and I believe it already offers some interesting properties compared to C > like > f.i. range checking and better type safety which could make a difference > for > security relevant projects like f.i. a web browser. The only thing that > would be > necessary for such a project was a *** well tested and ready-to-use > shipped *** > GUI toolkit ... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Jun 5 17:42:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 05 Jun 2015 08:42:17 -0700 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>, <5570D114.40104@marino.st> Message-ID: <20150605154217.AABC81A2062@async.async.caltech.edu> Jay K writes: >--_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > The FreeBSD4 users were surprisingly vocal surprisingly recently. > So I put some work into it. > I agree it is an old system. I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, but the 4.x config worked well. I've since upgraded to 10.0-RELEASE. And don't use it much anymore because posix threads (still) don't work right on FreeBSD. I spent a considerable effort to get them working right on Debian, which they now do, but didn't have the time to figure it out for a bunch of other OSes as well. It's really pretty nice to have them working correctly. They aren't perfect (a bit too much locking over garbage collection issues) but they don't ever seem to crash. On problems with lots of parallelism you do get some parallel speedup, even. Mika From jay.krell at cornell.edu Fri Jun 5 20:51:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 5 Jun 2015 18:51:13 +0000 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <5571688A.4060404@marino.st> References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st> Message-ID: This is kind of on purpose as Olaf said and kind of broken as you point out. We often build_standalone() to workaround the origin/rpath matte. We kind of fail here, in terms of library paths. I don't think the original system fully accounted for runpath. I think at the time, LD_LIBRARY_PATH might have been considered an acceptable solution. Elsewhere, with libtool, make install relinks to reset paths. We don't have that implemented. It is somewhat a good solution. - Jay > Date: Fri, 5 Jun 2015 11:14:50 +0200 > From: adacore at marino.st > To: wagner at elegosoft.com > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] install issues (stripping, permissions) > > On 6/5/2015 09:26, Olaf Wagner wrote: > > On Fri, 05 Jun 2015 08:10:08 +0200 > > John Marino wrote: > > > >> The other issue is that there were numerous files in pkg/ that were > >> executable and shouldn't have been. I used find/chmod to remove exec > >> perms for all except "cm3" file. I did this 1.5 years ago, I'm now > >> wondering how right it was. The current list: > >> > >> pkg/m3core/src/C/Common/Csetjmp.i3 > >> pkg/m3back/src/M3C.i3 > >> pkg/m3staloneback/AMD64_FREEBSD/m3back > >> pkg/cm3/AMD64_FREEBSD/cm3 > >> pkg/libdump/AMD64_FREEBSD/libdump > >> pkg/cmpfp/AMD64_FREEBSD/cmpfp > >> pkg/formsview/AMD64_FREEBSD/formsview > >> pkg/vorun/AMD64_FREEBSD/vorun > >> pkg/pkl-fonts/AMD64_FREEBSD/PklFonts > >> pkg/hack/AMD64_FREEBSD/dummy > >> pkg/test/AMD64_FREEBSD/test > >> > >> Should there be executables in pkg/ ? Is the chmod command then wrong? > >> I had to strips these as well to pass QA tests. > > > > Except for the i3-files these are all programs and should be executable. > > Okay, when I fixed this internally, I discovered something: > m3back, libdump, cmpfp, formsview, vorun, dummy, test all link to m3 > libraries but the rpath of $ORIGIN:$ORIGIN/../lib isn't enough. All of > these programs need an additional rpath to $ORIGIN/../../../lib for the > linker to find the libs. These all need to be fixed in repository I > suspect. > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jun 5 22:53:14 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 22:53:14 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st> Message-ID: <55720C3A.5070200@marino.st> On 6/5/2015 20:51, Jay K wrote: > This is kind of on purpose as Olaf said and kind of broken as you point out. > > We often build_standalone() to workaround the origin/rpath matte. > > We kind of fail here, in terms of library paths. well, to be blunt, it -is- broken and thus is a fail. > I don't think the original system fully accounted for runpath. > I think at the time, LD_LIBRARY_PATH might have been considered an > acceptable solution. If I wanted to go down a path like this, I'd use ldconfig, but my personal feeling if is ldconfig is required, the software is doing something wrong. > Elsewhere, with libtool, make install relinks to reset paths. We don't > have that implemented. > It is somewhat a good solution. I think it has to be a tweak at FreeBSD.common -- rather than use $ORIGIN/../lib for rpath, it needs a conditional statement to use $ORIGIN/../../../lib instead. I just don't know what the condition is. I'm also trying to figure out why rpath=$ORIGIN is needed. are there ever libraries in the same directory as cm3? ohn From jay.krell at cornell.edu Sat Jun 6 00:59:55 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 15:59:55 -0700 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55720C3A.5070200@marino.st> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> Message-ID: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> We could use origin, origin/../lib, origin/../../lib etc? But eventually a security problem to reach all around? Plain origin is in case the files are all together in one directory like on windows. Isn't it desirable to be buildable w/ system cc/ld and latest? Thank you for this work. - Jay On Jun 5, 2015, at 1:53 PM, John Marino wrote: > On 6/5/2015 20:51, Jay K wrote: >> This is kind of on purpose as Olaf said and kind of broken as you point out. >> >> We often build_standalone() to workaround the origin/rpath matte. >> >> We kind of fail here, in terms of library paths. > > well, to be blunt, it -is- broken and thus is a fail. > >> I don't think the original system fully accounted for runpath. >> I think at the time, LD_LIBRARY_PATH might have been considered an >> acceptable solution. > > If I wanted to go down a path like this, I'd use ldconfig, but my > personal feeling if is ldconfig is required, the software is doing > something wrong. > > >> Elsewhere, with libtool, make install relinks to reset paths. We don't >> have that implemented. >> It is somewhat a good solution. > > I think it has to be a tweak at FreeBSD.common -- rather than use > $ORIGIN/../lib for rpath, it needs a conditional statement to use > $ORIGIN/../../../lib instead. I just don't know what the condition is. > > I'm also trying to figure out why rpath=$ORIGIN is needed. are there > ever libraries in the same directory as cm3? > > ohn From jay.krell at cornell.edu Sat Jun 6 01:01:48 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 16:01:48 -0700 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> Message-ID: <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> Also "dist" means "distribution" means run on other machines, try not to be too specific to build machine. That is why not full paths. - Jay On Jun 5, 2015, at 3:59 PM, Jay wrote: > We could use origin, origin/../lib, origin/../../lib etc? But eventually a security problem to reach all around? Plain origin is in case the files are all together in one directory like on windows. > > Isn't it desirable to be buildable w/ system cc/ld and latest? > > Thank you for this work. > > - Jay > > On Jun 5, 2015, at 1:53 PM, John Marino wrote: > >> On 6/5/2015 20:51, Jay K wrote: >>> This is kind of on purpose as Olaf said and kind of broken as you point out. >>> >>> We often build_standalone() to workaround the origin/rpath matte. >>> >>> We kind of fail here, in terms of library paths. >> >> well, to be blunt, it -is- broken and thus is a fail. >> >>> I don't think the original system fully accounted for runpath. >>> I think at the time, LD_LIBRARY_PATH might have been considered an >>> acceptable solution. >> >> If I wanted to go down a path like this, I'd use ldconfig, but my >> personal feeling if is ldconfig is required, the software is doing >> something wrong. >> >> >>> Elsewhere, with libtool, make install relinks to reset paths. We don't >>> have that implemented. >>> It is somewhat a good solution. >> >> I think it has to be a tweak at FreeBSD.common -- rather than use >> $ORIGIN/../lib for rpath, it needs a conditional statement to use >> $ORIGIN/../../../lib instead. I just don't know what the condition is. >> >> I'm also trying to figure out why rpath=$ORIGIN is needed. are there >> ever libraries in the same directory as cm3? >> >> ohn From jay.krell at cornell.edu Sat Jun 6 02:20:44 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 17:20:44 -0700 Subject: [M3devel] rpath woes In-Reply-To: <20150605154217.AABC81A2062@async.async.caltech.edu> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> <5570D114.40104@marino.st> <20150605154217.AABC81A2062@async.async.caltech.edu> Message-ID: <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> Posix threads from C? From Modula-3? - Jay On Jun 5, 2015, at 8:42 AM, wrote: > Jay K writes: >> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> The FreeBSD4 users were surprisingly vocal surprisingly recently. >> So I put some work into it. >> I agree it is an old system. > > I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, > but the 4.x config worked well. > > I've since upgraded to 10.0-RELEASE. And don't use it much anymore > because posix threads (still) don't work right on FreeBSD. I spent a > considerable effort to get them working right on Debian, which they > now do, but didn't have the time to figure it out for a bunch of > other OSes as well. It's really pretty nice to have them working > correctly. They aren't perfect (a bit too much locking over garbage > collection issues) but they don't ever seem to crash. On problems > with lots of parallelism you do get some parallel speedup, even. > > Mika From mika at async.caltech.edu Sat Jun 6 02:32:10 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 05 Jun 2015 17:32:10 -0700 Subject: [M3devel] rpath woes In-Reply-To: <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> <5570D114.40104@marino.st> <20150605154217.AABC81A2062@async.async.caltech.edu> <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> Message-ID: <20150606003210.50F2E1A2065@async.async.caltech.edu> I think I've mentioned this on this mailing list about a hundred times, but... the pthreads implementation of Modula-3 threads doesn't work on most OSes. As far as I know the only OS where it is truly reliably working is AMD64_LINUX. I certainly haven't tested all targets. User-level threads work everywhere. Except for AMD64_LINUX I would not suggest putting anything important to use Modula-3 threads built on pthreads. They work... until they don't. Anyone wishing to work on this and fix it, please don't change the AMD64_LINUX code at all unless you know exactly, precisely what you are doing. It is very very difficult to debug the subtle bugs that get introduced. As far as I know there's no problem with C pthreads on FreeBSD. Mika Jay writes: >Posix threads from C? From Modula-3? > > - Jay > >On Jun 5, 2015, at 8:42 AM, wrote: > >> Jay K writes: >>> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >>> Content-Type: text/plain; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> The FreeBSD4 users were surprisingly vocal surprisingly recently. >>> So I put some work into it. >>> I agree it is an old system. >> >> I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, >> but the 4.x config worked well. >> >> I've since upgraded to 10.0-RELEASE. And don't use it much anymore >> because posix threads (still) don't work right on FreeBSD. I spent a >> considerable effort to get them working right on Debian, which they >> now do, but didn't have the time to figure it out for a bunch of >> other OSes as well. It's really pretty nice to have them working >> correctly. They aren't perfect (a bit too much locking over garbage >> collection issues) but they don't ever seem to crash. On problems >> with lots of parallelism you do get some parallel speedup, even. >> >> Mika From adacore at marino.st Sat Jun 6 07:53:40 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 07:53:40 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> Message-ID: <55728AE4.4030301@marino.st> On 6/6/2015 01:01, Jay wrote: > Also "dist" means "distribution" means run on other machines, try not to be too specific to build machine. That is why not full paths. > > - Jay > > On Jun 5, 2015, at 3:59 PM, Jay wrote: >>> I think it has to be a tweak at FreeBSD.common -- rather than use >>> $ORIGIN/../lib for rpath, it needs a conditional statement to use >>> $ORIGIN/../../../lib instead. I just don't know what the condition is. >>> >>> I'm also trying to figure out why rpath=$ORIGIN is needed. are there >>> ever libraries in the same directory as cm3? >>> >>> ohn I think you misunderstood. I know why "$ORIGIN/../lib" is there. I don't know why "$ORIGIN" is there. There are no libraries in /usr/local/cm3/bin, so why is this directory specified? John From adacore at marino.st Sat Jun 6 07:57:50 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 07:57:50 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> Message-ID: <55728BDE.8030702@marino.st> On 6/6/2015 00:59, Jay wrote: > We could use origin, origin/../lib, origin/../../lib etc? But > eventually a security problem to reach all around? Plain origin is in > case the files are all together in one directory like on windows. I'm not really a fan of just adding every possible rpath. In this case, it's either $ORIGIN/../lib or $ORIGIN/../../../lib, but together is never correct (as the other mail indicates, $ORIGIN by itself also seems to be always wrong) > > Isn't it desirable to be buildable w/ system cc/ld and latest? On DragonFly - yes On FreeBSD - no. FreeBSD is stuck with either ancient gcc (4.2.1), ancient binutils (2.17) or clang. In every case, it needs to use newer GCC and binutils from ports to build M3. (and every newer GCC) John From jay.krell at cornell.edu Sat Jun 6 10:47:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 6 Jun 2015 08:47:25 +0000 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55728BDE.8030702@marino.st> References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st>, , <55720C3A.5070200@marino.st>, <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com>, <55728BDE.8030702@marino.st> Message-ID: $ORIGIN is "in case" we restructure Unix install to look like Windows install -- moving lib on top of bin. I realize you'd like just the minimum required, not hypothetical values for the future. Regarding gcc/ld -- isn't it desirable to have either/both work? Are there command lines that work with both and provide the same meaning? Does our stuff not work with older gcc/ld? Do ports in general avoid system gcc/ld? Does clang work? I intend to try. - Jay > Date: Sat, 6 Jun 2015 07:57:50 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] install issues (stripping, permissions) > > On 6/6/2015 00:59, Jay wrote: > > We could use origin, origin/../lib, origin/../../lib etc? But > > eventually a security problem to reach all around? Plain origin is in > > case the files are all together in one directory like on windows. > > I'm not really a fan of just adding every possible rpath. In this case, > it's either $ORIGIN/../lib or $ORIGIN/../../../lib, but together is > never correct (as the other mail indicates, $ORIGIN by itself also seems > to be always wrong) > > > > > Isn't it desirable to be buildable w/ system cc/ld and latest? > > On DragonFly - yes > On FreeBSD - no. > > FreeBSD is stuck with either ancient gcc (4.2.1), ancient binutils > (2.17) or clang. In every case, it needs to use newer GCC and binutils > from ports to build M3. (and every newer GCC) > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jun 6 10:48:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 6 Jun 2015 08:48:41 +0000 Subject: [M3devel] rpath woes In-Reply-To: <20150606003210.50F2E1A2065@async.async.caltech.edu> References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>,<5570D114.40104@marino.st> , <20150605154217.AABC81A2062@async.async.caltech.edu>, <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com>, <20150606003210.50F2E1A2065@async.async.caltech.edu> Message-ID: I thought these problems were all fixed by now. :( Maybe cooperative suspend will help. I know that seems like a non-sequiter, but I know that SuspendThread and GetThreadContext on Windows don't do what you'd expect and as a result our x86-on-amd64 code isn't correct. - Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > Date: Fri, 5 Jun 2015 17:32:10 -0700 > From: mika at async.caltech.edu > > I think I've mentioned this on this mailing list about a hundred times, but... > > the pthreads implementation of Modula-3 threads doesn't work on most OSes. > > As far as I know the only OS where it is truly reliably working is AMD64_LINUX. > > I certainly haven't tested all targets. > > User-level threads work everywhere. > > Except for AMD64_LINUX I would not suggest putting anything important to > use Modula-3 threads built on pthreads. They work... until they don't. > > Anyone wishing to work on this and fix it, please don't change the > AMD64_LINUX code at all unless you know exactly, precisely what you > are doing. It is very very difficult to debug the subtle bugs that > get introduced. > > As far as I know there's no problem with C pthreads on FreeBSD. > > Mika > > Jay writes: > >Posix threads from C? From Modula-3? > > > > - Jay > > > >On Jun 5, 2015, at 8:42 AM, wrote: > > > >> Jay K writes: > >>> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ > >>> Content-Type: text/plain; charset="iso-8859-1" > >>> Content-Transfer-Encoding: quoted-printable > >>> > >>> The FreeBSD4 users were surprisingly vocal surprisingly recently. > >>> So I put some work into it. > >>> I agree it is an old system. > >> > >> I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, > >> but the 4.x config worked well. > >> > >> I've since upgraded to 10.0-RELEASE. And don't use it much anymore > >> because posix threads (still) don't work right on FreeBSD. I spent a > >> considerable effort to get them working right on Debian, which they > >> now do, but didn't have the time to figure it out for a bunch of > >> other OSes as well. It's really pretty nice to have them working > >> correctly. They aren't perfect (a bit too much locking over garbage > >> collection issues) but they don't ever seem to crash. On problems > >> with lots of parallelism you do get some parallel speedup, even. > >> > >> Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sat Jun 6 14:02:29 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 14:02:29 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st>, , <55720C3A.5070200@marino.st>, <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com>, <55728BDE.8030702@marino.st> Message-ID: <5572E155.2040408@marino.st> On 6/6/2015 10:47, Jay K wrote: > $ORIGIN is "in case" we restructure Unix install to look like Windows > install -- moving lib on top of bin. > I realize you'd like just the minimum required, not hypothetical values > for the future. Since it's patched anyway, I'll just remove $ORIGIN then. Obviously I'm going to vote against a lowest-common-denomination approach of dumping everything into one directory. A better way is have the build system support either. > Regarding gcc/ld -- isn't it desirable to have either/both work? Given scarce developer resources (and small user population), I recommend that you target supporting ports rather than try to support multiple configurations. Unless you are really committed to supported platforms that have been EOL for years (basically FreeBSD 8.x and lower) then I don't see a big ROI on supporting more than what ports requires. To specifically answer, I'd say "no". FreeBSD 9, 10, 11 all have different base compiler configurations. It's not worth trying to support the quirks of each one. > Are there command lines that work with both and provide the same meaning? I'm not sure, but since ports works on FreeBSD 9 and later, I am not that motivated to do the extensive testing necessary to answer that. > Does our stuff not work with older gcc/ld? > Do ports in general avoid system gcc/ld? More and more. The freebsd base assembler is ancient, so often a new gas is required which brings in the new ld and forces that ports GCC be used (if not base clang) > Does clang work? > I intend to try. I expect it to work on the C-backend. John From mika at async.caltech.edu Sat Jun 6 18:50:23 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 06 Jun 2015 09:50:23 -0700 Subject: [M3devel] the need for cooperative suspend In-Reply-To: References: <20150502155713.29DCC1A2062@async.async.caltech.edu> Message-ID: <20150606165023.78AEC1A2066@async.async.caltech.edu> Can the thread tester survive with the most paranoid options on OS X? I don't believe it until I've seen it (based on what we went through for Linux, I would say I'm just being prudent). Run it with: "-iters 100000 -n 20 -tests ALL" I remember the thread tester died on FreeBSD last time I tried, and there have been no relevant checkins since that I know of. Here's what happens when I run it on my FreeBSD. CM3 is not up to date, in case someone fixed something I'm not aware of, so don't draw too many conclusions until someone has tried it on a fresh install: ..........laziest thread is 1433608799/9/1 (tests: read 10/6/6 nxread 26/6/4 tryexcept 67/53/38 fork 1433608799/2/2 forktoomuch 1433608799/9/3 alloc 1433608799/37/1 creat 10/7/7 lock 1433608799/68/38) *** *** runtime error: *** Segmentation violation - possible attempt to dereference NIL *** pc = 0x4387e8 = Move + 0x6a in ../src/runtime/common/RTCollector.m3 *** Abort (79)pluto:/tmp> Anyone who's interested in trying their own favorite Modula-3 installation, this is easy to do: cm3/m3-libs/m3core/tests/thread/AMD64_FREEBSD/threadtest -iters 100000 -n 20 -tests ALL Ignore the warnings about "thread being starved or deadlocked". Those should really be an option and not on by default: as far as I know the threading library makes no claim of fairness, so I don't think one should expect it by default. Mika Antony Hosking writes: >OS X is stable as well as Linux, as far as I know. >Mika, what misbehaviors are you seeing? > >Sent from my iPad > >> On May 2, 2015, at 11:57 AM, mika at async.caltech.edu wrote: >>=20 >> Jay,=20 >>=20 >> Can you explain precisely what you mean by "suspend"? >>=20 >> A thread is suspended by ... a timer expiration? Another thread at higher= > priority needing resources? >>=20 >> I'm thinking about a tight loop "for(;;) ;"... how do you stop a thread fr= >om running that (it could have >> been coded in assembly...)? >>=20 >> Jay K writes: >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_ >>> Content-Type: text/plain; charset=3D"iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>>=20 >>> We really need to move away from preemptive suspend. >>>=20 >>> 1) It is a large fraction of the target-dependent code in m3core. >>> =3D20 >>>=20 >>> 2) It doesn't really work in "wow64"=3D2C the x86-on-amd64 Windows system= >. >>> The SuspendThread + GetThreadContext does not actually reliably give you >>> current and coherent context. >>> I'm not sure about x86-on-ia64. >>> =3D20 >>>=20 >>> The context can be old. >>> The context can be mid-update. Which might be ok=3D2C depending on what t= >he =3D >>> updates are. >>> LIke if Eip and Esp are being updated=3D2C we don't care about Eip anyway= >. >>> =3D20 >>>=20 >>> You can find out if either is the case=3D2C and let the thread run longer= >=3D2C >>> but being in a long running syscall I believe is reported as possibly >>> invalid even though it is valid. >>> =3D20 >>> =3D20 >>> NT/native is ok. >>> =3D20 >>>=20 >>> We should just use cooperative suspend and not worry about it. >>> =3D20 >>> =3D20 >>> Thanks=3D2C >>> - Jay >>>=20 >>>=20 >>> =3D20 >>> =3D >>>=20 >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_ >>> Content-Type: text/html; charset=3D"iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>>=20 >>> >>> >>> >>>
We really need to move awa= >y from=3D >>> preemptive suspend.

1) It is a large fraction of the target-depend= >e=3D >>> nt code in m3core.
 =3D3B

2) It doesn't really work in "wow= >64"=3D >>> =3D2C the x86-on-amd64 Windows system.
The SuspendThread + GetThreadCo= >ntex=3D >>> t does not actually reliably give you
current and coherent context.>I=3D >>> 'm not sure about x86-on-ia64.
 =3D3B

 =3D3BThe context= > can b=3D >>> e old.
 =3D3BThe context can be mid-update. Which might be ok=3D2C= > depen=3D >>> ding on what the updates are.
 =3D3BLIke if Eip and Esp are being u= >pda=3D >>> ted=3D2C we don't care about Eip anyway.
 =3D3B

You can fin= >d out =3D >>> if either is the case=3D2C and let the thread run longer=3D2C
but bein= >g in a=3D >>> long running syscall I believe is reported as possibly
invalid even th= >o=3D >>> ugh it is valid.
 =3D3B
 =3D3B
NT/native is ok.
 = >=3D3B>>>
We should just use cooperative suspend and not worry about it.
&n= >bs=3D >>> p=3D3B
 =3D3B
Thanks=3D2C
 =3D3B- Jay


 =3D= >3B
=3D >>>
>>> =3D >>>=20 >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_-- From adacore at marino.st Sat Jun 6 23:32:52 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 23:32:52 +0200 Subject: [M3devel] Follow up - FreeBSD port Message-ID: <55736704.6030900@marino.st> Based on recent discussions, I altered the FreeBSD port as follows: 1) I removed the rpath=$ORIGIN flag 2) I limited the de-execution to the executable .i3 files 3) I moved the executable pkg programs to bin, then added softlinks to where the were originally installed. The last one enables those programs to work again, even from the pkg/*/AMD64_FREEBSD/ locations. I could not figure out a way to dynamically change the rpath so I just decided to use $ORIGIN/../lib for everything an make it work. It has a side benefit of eliminating the large, duplicate cm3 at pkg/cm3/AMD64_FREEBSD/cm3 (replaced by symlink). I think the port is in pretty good shape right now. Most of the programs work, but cm3ide core dumps a lot. It also seems to be querying a database that doesn't exist (pgsql installed but port doesn't do anything with it). Anything that tries to use TCP seems to fail. John http://www.freshports.org/commit.php?category=lang&port=modula3&files=yes&message_id=201506062130.t56LUF55083786 at svn.freebsd.org From mika at async.caltech.edu Sun Jun 7 08:05:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 06 Jun 2015 23:05:17 -0700 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <55736704.6030900@marino.st> References: <55736704.6030900@marino.st> Message-ID: <20150607060517.0FF6E1A2067@async.async.caltech.edu> John Marino writes: ... >Anything that tries to use TCP seems to fail. ... Might be a wild goose chase.. but I seem to remember having issues if the reverse and forward DNS of your machine don't match up, or if there is no reverse DNS... or something like that. Otherwise, TCP should work just fine. Mika From hendrik at topoi.pooq.com Sun Jun 7 14:47:12 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 7 Jun 2015 08:47:12 -0400 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <20150607060517.0FF6E1A2067@async.async.caltech.edu> References: <55736704.6030900@marino.st> <20150607060517.0FF6E1A2067@async.async.caltech.edu> Message-ID: <20150607124712.GA1045@topoi.pooq.com> On Sat, Jun 06, 2015 at 11:05:17PM -0700, mika at async.caltech.edu wrote: > John Marino writes: > ... > >Anything that tries to use TCP seems to fail. > ... > > Might be a wild goose chase.. but I seem to remember having issues if > the reverse and forward DNS of your machine don't match up, or if there > is no reverse DNS... or something like that. That's not TCP doing that as far as I know. Even DNS is built on top of TCP and IP, and TCP knows nothing of it. However, a lot of email servers perform such a check to block some forms of malfeasance. Possibly some other systems built on top of TCP do that as well. > > Otherwise, TCP should work just fine. > > Mika From mika at async.caltech.edu Sun Jun 7 18:03:50 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 07 Jun 2015 09:03:50 -0700 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <20150607124712.GA1045@topoi.pooq.com> References: <55736704.6030900@marino.st> <20150607060517.0FF6E1A2067@async.async.caltech.edu> <20150607124712.GA1045@topoi.pooq.com> Message-ID: <20150607160350.D5C1F1A2066@async.async.caltech.edu> Hendrik Boom writes: >On Sat, Jun 06, 2015 at 11:05:17PM -0700, mika at async.caltech.edu wrote: >> John Marino writes: >> ... >> >Anything that tries to use TCP seems to fail. >> ... >> >> Might be a wild goose chase.. but I seem to remember having issues if >> the reverse and forward DNS of your machine don't match up, or if there >> is no reverse DNS... or something like that. > >That's not TCP doing that as far as I know. Even DNS is built >on top of TCP and IP, and TCP knows nothing of it. > >However, a lot of email servers perform such a check to block some >forms of malfeasance. Possibly some other systems built on top of >TCP do that as well. > >> >> Otherwise, TCP should work just fine. >> >> Mika Well, yeah, it's obviously not TCP itself---don't think anyone claimed as much. It's something about the Modula-3 code that wraps TCP socket code making assumptions about the configuration of the host it's running on (that that host is configured in a "reasonable way" for a workstation in an industrial research lab, perhaps...) I remember running into it a few times and don't know if I (or anyone) ever fixed it in the distribution. Mika From rodney_bates at lcwb.coop Sun Jun 7 19:32:59 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 07 Jun 2015 12:32:59 -0500 Subject: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? In-Reply-To: <55708E2B.4040607@elstel.org> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> <55708E2B.4040607@elstel.org> Message-ID: <5574804B.5030304@lcwb.coop> On 06/04/2015 12:43 PM, Elmar Stellnberger wrote: > Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > Am 04.06.15 um 18:35 schrieb Rodney M. Bates: >> >> >> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: >>> Am 03.06.15 um 03:58 schrieb Jay K: >>>> We cannot cross from 32bit host to 64bit target. >>>> >>>> >>>> Ok to commit this? >>>> >>>> >>> >>> Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. >>> To me this is just another argument for a language specified value range of INTEGER. >>> It makes certain things easier. >>> >>> My suggestion was: >>> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) >>> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) >>> BITS 64 FOR INTEGER would only work on x64 systems. >>> >> >> What you want is already pretty much here. Just declare your desired type yourself: >> >> TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; > This should be good news. However my suggestion was to make Int32.T the default > as this will be somewhat faster for the average application which is data intensive (just > think of image processing) which will never be rewritten to use Int32.T instead of INT. > All of our current and old programs being prepared to run on a 32bit as well as a 64bit > target should compile and run without any problem. There is one single exception > where the compiler would throw an error: when using INTEGER for address arithmetics > in unsafe modules (error is: can only LOOPHOLE between types of the same size, or > some way similar as I have already tested this). There we would need the so called > targetint/offset. This would be a very biased thing to do. The only people who would benefit would be those who know for certain that they want *every* integer variable to have 32-bit range, on both 32-bit and 64-bit machines. Without debating the wisdom or likelihood of this, a quite easy string search/replace on "INTEGER" would accomplish the same thing. But everybody else would have to vet their integers one by one, to see which they wanted always 32 and which they wanted native size. It would break lots of code for the unlucky (majority, I would guess), while providing only speed improvements for the favored. I have worked on lots of code in the M3 distribution itself that would break. > If I understand things right the following two things would need to get implemented > to allow a default int size of 32 for existing applications (be it the default or just an > additional compiler option as the size of WIDECHAR already is): > * automatic range adaption for packed ints (i.e. those with BITS .. FOR) No, your are still trying to misuse BITS to control the value range. Subranges do this just fine. Use the mechanism the language already provides. BITS is for manual control of record/object/array layout only, not value range. > * allowing non 32bit alignment of INTs in memory (as is already the case for WIDE[CHAR]s) INTEGERs (and subranges thereof) are currently no different from WIDECHARS, in regard to alignment. They get the same treatment as anything else. For BITS types inside a record/object/array, the language lets the compiler choose what it will accept, but I am reasonably sure all existing M3 compiler variations, for all targets, will lay things out as you ask (including misaligned) and will correctly access them, as long as no item crosses a native word boundary. Outside records/objects/arrays, the compiler just aligns as needed. >> >> It will always have this range and occupy 32 bits, regardless of the native word size. Pickles >> will always save and reread it with exactly this range, even between machines of different >> word size. > So the target size of the pickle is already determined by the respective INT subrange, right? Defining what pickles do/should do is fraught with pitfalls. Here's what we now have: For (almost) all types, pickles preserve the properties of the M3 type. The bounds of a subrange type are static constant numbers, and are evaluated at compile time on the compiling machine. So [0..16_3FFFFFFF] is the same type on either size of machine, but [0..LAST(INTEGER) DIV 2] is not. On a 32-bit machine, [0..LAST(INTEGER) DIV 2] evaluates to, and thus means [0..16_3FFFFFFF], but on a 64-bit machine, it means [0..16_3FFFFFFFFFFFFFFF], a different type. So, for example, you could exchange pickled values between 32-bit and 64-bit machines if the field/element of the containing record/object/array type were written as [0..16_3FFFFFFF] on both machines, or if it was written as [0..LAST(INTEGER) DIV 2] on the 32-bit machine, and [0..16_3FFFFFFF] on the 64, but not if both had coded the type as [0..LAST(INTEGER) DIV 2]. INTEGER, CARDINAL, LONGINT, and LONGCARD are exceptions. Each of these is treated as the same M3 type, regardless of native word size and upper bound. So pickles will copy a field/element of any these types with a size change, when written and read on different sized machines. The size change does the more-or-less obvious sign-extend one way and raises an exception if the value won't fit, when going the other way. The floating types are not handled by pickles across machines with different floating formats. If the format is the same, (and one pickles knows exists), they will work, otherwise, they will raise an exception. It would be nice to implement this, but apparently, nobody has had a sufficient need. I am no numerical analyst, but I think defining good usable rules for this would call for the services of someone who is. > Something that should to my believe be well documented as one could easily like to extend > an enum with 255 members to more or equal than 256 an then wonder why the new an the > old file formats become incompatible. To circumvent such a problem we would likely need > to allow BITS 8*anything FOR INTEGER and take the bitcount specified in BITS for in order to > determine the target size of an object instead of semi-automatically readjusting it all the > time. This one really gets into a tar pit. Pickles encode the type of a value as a "fingerprint", which is a 64-bit has of the M3 type structure. A copy of this goes into the pickle for every type it contains. Meanwhile, the compiler puts copies of the fingerprints of all types in the program into static global readonly data. Pickle read uses this to look or a type defined in the reading program with the same signature, which means it must be exactly the same type, for the read to succeed. Allowing similar but unequal types would very difficult to define even what differences would be tolerated. This get very complicated very fast. Remember, there is no guarantee that the reading program is the same program as the writing program, or has much similarity to it at all. Pickles requires only that the reading program have a defined reference type that is structurally identical to that of the object had in the writing program. Allowing to write a field of type {Red, Green} in a pickle and read it into a program that has {Red, Green, Blue} would get very messy. Remember that pickles does not handle scalar values, just heap objects. So we would have to be defining some recursive rule about record types whose corresponding field types were each sufficiently "similar", etc, etc. And to implement, the pickle format would have to be completely reworked to contain full type structural descriptions, not just signatures. >> >> The only thing that will differ from 32- to 64-bit machine is the size of intermediate results in >> expressions. If you are already _not_ being careful about avoiding overflows, there are probably >> cases where the results of silent overflows would differ with machine word size. Not sure what >> you would really want here. The unsigned arithmetic operations in Word are explicitly defined >> to silently do binary twos-complement wrap-around when overflows happen. The language does not >> define this for the builtin signed operators. In code where I care about overflows, I never assume >> this. It is probably machine-dependent. > Yes, I believe it would be good like this. No real 32bit INT with 32bit arithmetics but just restricting > the range for Int32.T when it gets written into memory. >> >> Or, you can import it: Int32.T. is already declared, in m3core, with this range. >> >> However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is almost never what you really want. >> The only place the BITS specification makes any difference is when you declare a record or object >> field of this type. In all other situations, BITS 32 FOR .. has _no effect_. The only purpose of >> BITS is so you can force the layout of a record. In that case, since the range needs exactly 32 >> bits anyway, even if you omit the BITS, a field of this type will still always occupy 32 bits. > Nonetheless it is currently not possible to declare a variable of a 32bit type globally: > > TYPE Pixel = BITS 32 FOR INTEGER; > > unluckily does not work as we can have no global variable of type Pixel. No, you have misunderstood. Any type BITS n OF T can be used anywhere T can be used, including global variables, local variables, parameters, etc. It's just that, except for a field/element, the addition of BITS... will have no effect, in other words, it won't change that number of bits or alignment the compiler actually allocates, from what the compiler would do if the type were given as just T. It will not make the declaration illegal. My recommendation not to use BITS in a named TYPE declaration was only because: 1) Outside of a record/object/array, it will make no difference anyway, and 2) Inside of a record/object/array, the effect it has can and often will be different every time, so you have to think about each such field/element and separately judge whether the BITS specification is right for that particular case. And that depends on what comes before it in the record/object/array. Also, there is full assignability between BITS n FOR T and T. So you don't have to put explicit conversions in. The one case there is not such assignability is between BITS n FOR T and BITS m FOR T, with n#m. So copying between two record fields of the same base type but different explicit BITS specifications of different sizes would require explicit conversion. > This is not only un-orthogonal but also a very practical problem; Suggesting that we call > procedures with a parameter of type pixel we will never be able to change that type f.i. to > a packed record with members red, green, blue alpha if there are places in our main > program where we need to declare VAR pix1, pix2 : INTEGER just because Pixel is a packed > type being refused as global variable. > >> >> But, with the BITS specification, the compiler is not allowed to insert alignment padding, (if it >> were, you could not fully specify the layout) and if the field comes at a place that is not 32-bit >> aligned, you will either get a compile-time error or maybe a compiler will accept it and handle >> unaligned field access (it is not required to--ours does not), neither of which is probably what >> you want. Without BITS, the compiler will pad as needed. With BITS, you will have to insert explicit >> padding for alignment yourself as needed, which can be different for every field of this type of every >> record/object. And later adding/removing/changing any prior field can upset it and require you to >> rework the padding. Moreover, I am sure there are simple cases where the required padding differs >> between 32-bit and 64-bit machines. > On the long term I would personally even advocate an ALIGN directive: > f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END; > > It would be very handy to have this for tapping the full power of the vectorization units > of current processors. Manually re-aligning heap allocated objects in an unsafe module > is not a very satisfying alternative (no local/global/stack allocation, memory waste and > thus a certain performance loss if there are many wholes). However this would possibly > also make changes to our garbage collector necessary as it currently only guarantees > word alignment. Finally it would also be possible to write ALIGN BITSIZE(TARGETINT) .... > > >> >> Having been bitten by this more than once, I now only put BITS..FOR as the anonymous type of a specific >> field, never in a named type to be used in multiple places. > A less bitchy and more orthogonal Modula-3 compiler supporting integer globals and > locals of non word size and alignment would be a real pleasure, do you consent Rodney? >> >> (Actually, BITS does the same thing when used as the type of an array element, but cases where this >> matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >> needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >> fits in a single machine word. I do have a couple of such cases.) >> >>> The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value >>> range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for >>> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). >>> >>> Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in >>> here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In >>> a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same >>> tends to be much slower). >>> >>> Regards, >>> Elmar >>> >> > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 9 03:44:07 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 8 Jun 2015 18:44:07 -0700 Subject: [M3devel] m3cg right shift signed? Message-ID: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> (Not at propercomputer but sending anyway.) Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? - Jay From hendrik at topoi.pooq.com Tue Jun 9 14:38:29 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 9 Jun 2015 08:38:29 -0400 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> Message-ID: <20150609123829.GA7874@topoi.pooq.com> On Mon, Jun 08, 2015 at 06:44:07PM -0700, Jay wrote: > (Not at propercomputer but sending anyway.) > > Is right shift of a signed type meant to be supported and well > defined in the IR? I thought not, and assert so in the C backend. But > it occurs now, from m3front/cg/set_range or such. Maybe change that > to not? Surely unsigned types are adequate here? Signed and unsigned right shifts are different on most hardware that supports them. Signed shifts do sign extension, so that the shifts of negative numbers remain negative. Unsigned shifts do zero extension. There are (once were?) machines where shifts are (were?) faster than division by powers of two. Not sure how this relates to whatever in Modula 3 might or might not require them, though. -- hendrik From jay.krell at cornell.edu Tue Jun 9 17:42:59 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 15:42:59 +0000 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <20150609123829.GA7874@topoi.pooq.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com>, <20150609123829.GA7874@topoi.pooq.com> Message-ID: Division is among the slowest instructions that operates on integers even on modern hardware. It is/was frequently completely absent, though maybe those days are past. 6502 and 65816 had no hardware multiply or divide, nor maybe signed shift. - Jay > Date: Tue, 9 Jun 2015 08:38:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cg right shift signed? > > On Mon, Jun 08, 2015 at 06:44:07PM -0700, Jay wrote: > > (Not at propercomputer but sending anyway.) > > > > Is right shift of a signed type meant to be supported and well > > defined in the IR? I thought not, and assert so in the C backend. But > > it occurs now, from m3front/cg/set_range or such. Maybe change that > > to not? Surely unsigned types are adequate here? > > Signed and unsigned right shifts are different on most hardware that > supports them. Signed shifts do sign extension, so that the shifts of > negative numbers remain negative. Unsigned shifts do zero extension. > > There are (once were?) machines where shifts are (were?) faster > than division by powers of two. > > Not sure how this relates to whatever in Modula 3 might or might not > require them, though. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 9 18:05:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 09 Jun 2015 11:05:51 -0500 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> Message-ID: <55770EDF.4030606@lcwb.coop> I'm a little unclear what you are asking. I see several places in M3CG_Ops.i3 which call for sign-extension, e.g., load, widen, extract* and any way of pushing on the IR stack. While signed right shift is clearly a good primitive to sign-extend, all these IR ops leave leave this to the code generator, which is good, because the available machine opcodes will be target dependent. As for set_range, this too leaves it to the code generator when the word whose bits are to be set is 32 or 64 bits. For smaller, CG lowers this itself, using more primitive bit-twiddling operators. But it looks like the shifts it generates must be zero-filling, to work right. As for the shift IR operators themselves, M3CG_Ops.i3 defines them as the same as the shifts in Word, none of which is sign-filling. On 06/08/2015 08:44 PM, Jay wrote: > (Not at propercomputer but sending anyway.) > > Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? > > - Jay > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 9 20:30:17 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 18:30:17 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? Message-ID: Is there any interest in maintaining 1) gcc/m3cg/parse.c support for gcc 4.3/4.5/4.6? It is slightly messy. 1b) gcc 4.2? This is where hypothetical ARM_DARWIN support is, last I checked, years ago. 2) gcc-4.3/4.5/4.6 in-tree? Or remove them to keep checkouts smaller? There was complaint as to our source tree size, and these directories do add a lot of size for little gain. They were historically useful to transition versions. I don't think it was wrong to have the temporary growth. All targets except for ARM_DARWIN default to gcc 4.7. Do people often/ever go back and compare/debug? Or want to retain that ability, with the current ease? Or ok to get the files from history, for that rare-to-never event? If we do maintain the gcc backend, I would likely import as gcc-5.1 etc., and recreate the problem, but it can be a "rolling" problem and fix. But I'm not keen on maintaining the gcc backend anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 20:32:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 18:32:03 +0000 Subject: [M3devel] ARM_DARWIN? Message-ID: ARM_DARWIN has never fully worked.I got to the point of c_compile/assemble/link cm3 on the target and then running cm3 failed.I don't remember why. I no longer have a system/toolset setup to test that. Does anyone have an iPhone or iPad (or simulator?) setup with ssh access and native C compiler, assembler, and linker?If so I'd like access to resume/finish that work. And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider deleting gcc-apple. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Jun 9 20:58:20 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 9 Jun 2015 11:58:20 -0700 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: Message-ID: I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that do the trick? On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: > ARM_DARWIN has never fully worked. > I got to the point of c_compile/assemble/link cm3 on the target and then > running cm3 failed. > I don't remember why. > > > I no longer have a system/toolset setup to test that. > > > Does anyone have an iPhone or iPad (or simulator?) setup with ssh access > and native C compiler, assembler, and linker? > If so I'd like access to resume/finish that work. > > > And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider > deleting gcc-apple. > > > Thank you, > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 21:29:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 19:29:41 +0000 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: , Message-ID: Yes that hardware will likely do -- if it can be setup for ssh and w/ native cc/as/ld.But I'd really rather someone else do the setup work. - Jay Date: Tue, 9 Jun 2015 11:58:20 -0700 Subject: Re: [M3devel] ARM_DARWIN? From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that do the trick? On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: ARM_DARWIN has never fully worked.I got to the point of c_compile/assemble/link cm3 on the target and then running cm3 failed.I don't remember why. I no longer have a system/toolset setup to test that. Does anyone have an iPhone or iPad (or simulator?) setup with ssh access and native C compiler, assembler, and linker?If so I'd like access to resume/finish that work. And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider deleting gcc-apple. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Jun 9 21:51:20 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 9 Jun 2015 12:51:20 -0700 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: Message-ID: Well, according to this < http://guides.macrumors.com/SSH_into_your_iPod_touch_(Windows) > it's jailbreak and then install OpenSSH, then access over wifi. I can do that easily enough. The bigger issue for remove development would be resetting the device. It's battery powered so it requires a physical access to reset. On Tue, Jun 9, 2015 at 12:29 PM, Jay K wrote: > Yes that hardware will *likely *do -- if it can be setup for ssh and w/ > native cc/as/ld. > But I'd really rather someone else do the setup work. > > - Jay > > > > ------------------------------ > Date: Tue, 9 Jun 2015 11:58:20 -0700 > Subject: Re: [M3devel] ARM_DARWIN? > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > > > I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that > do the trick? > > On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: > > ARM_DARWIN has never fully worked. > I got to the point of c_compile/assemble/link cm3 on the target and then > running cm3 failed. > I don't remember why. > > > I no longer have a system/toolset setup to test that. > > > Does anyone have an iPhone or iPad (or simulator?) setup with ssh access > and native C compiler, assembler, and linker? > If so I'd like access to resume/finish that work. > > > And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider > deleting gcc-apple. > > > Thank you, > - Jay > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Tue Jun 9 22:02:45 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 09 Jun 2015 22:02:45 +0200 Subject: [M3devel] cm3: what are *.mc files Message-ID: <55774665.8060303@elstel.org> What are *.mc - files? They appear in TARGET - directories; most of them are just called _m3main.mc but some of them have other names. I ask because I am writing a program which should recognize and clear object files. It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. Usually files of a specific type start with a 32bit magic; however the mc files all have different starting sequences. Is there still a straight forward way to recognize an .mc file just by its binary content? From jay.krell at cornell.edu Tue Jun 9 22:12:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 20:12:51 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <55774665.8060303@elstel.org> References: <55774665.8060303@elstel.org> Message-ID: I believe before the m3cc backend existed, these were actually C code. (i.e. C backend used to be the one and only) Then they morphed into the intermediate representation fed to cm3cgwhich is what they are now. See here: https://github.com/modula3/cm3/tree/master/m3-sys/m3middle/src/M3CG_BinWr.m3 and m3-sys/m3cc/gcc/gcc/m3cg/parse.c m3cgcat can dump the binary form into text. _m3main.mc is the intermediate file that holds main.There is a configuration option to use it, vs. _m3main.c.I switched to _m3main.c. This is perhaps more portable if using C++ anywhere andwant to get the constructors/destructors run (though all modern implementationsdon't care really). Try cm3 -keep to see them all. i.e. normally they get deleted. - Jay > Date: Tue, 9 Jun 2015 22:02:45 +0200 > From: estellnb at elstel.org > To: m3devel at elegosoft.com > Subject: [M3devel] cm3: what are *.mc files > > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear > object files. > It does not seem to be sufficient to check for uppercase directories > which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by > its binary content? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 22:14:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 20:14:03 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: References: <55774665.8060303@elstel.org>, Message-ID: ps: foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io again, see cm3 -keep, err better yet, cm3 -keep -verbose You can see it running cm3cg and as and rm. - Jay From: jay.krell at cornell.edu To: estellnb at elstel.org; m3devel at elegosoft.com Date: Tue, 9 Jun 2015 20:12:51 +0000 Subject: Re: [M3devel] cm3: what are *.mc files I believe before the m3cc backend existed, these were actually C code. (i.e. C backend used to be the one and only) Then they morphed into the intermediate representation fed to cm3cgwhich is what they are now. See here: https://github.com/modula3/cm3/tree/master/m3-sys/m3middle/src/M3CG_BinWr.m3 and m3-sys/m3cc/gcc/gcc/m3cg/parse.c m3cgcat can dump the binary form into text. _m3main.mc is the intermediate file that holds main.There is a configuration option to use it, vs. _m3main.c.I switched to _m3main.c. This is perhaps more portable if using C++ anywhere andwant to get the constructors/destructors run (though all modern implementationsdon't care really). Try cm3 -keep to see them all. i.e. normally they get deleted. - Jay > Date: Tue, 9 Jun 2015 22:02:45 +0200 > From: estellnb at elstel.org > To: m3devel at elegosoft.com > Subject: [M3devel] cm3: what are *.mc files > > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear > object files. > It does not seem to be sufficient to check for uppercase directories > which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by > its binary content? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 10 02:21:23 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 09 Jun 2015 19:21:23 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <55774665.8060303@elstel.org> References: <55774665.8060303@elstel.org> Message-ID: <55778303.3010009@lcwb.coop> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear object files. > It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by its binary content? > They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, or 16_FD 16_10 16_01, produced by a very recent head compiler. Ignore the 4th byte. > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 10 11:12:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:12:06 +0000 Subject: [M3devel] AMD64_DARWIN config Message-ID: I'd like to bring this back to what it was: diff --git a/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN b/m3-sys/cminstall/src/config-no-install/AMD64_DARWINindex c30dd35..8fa0a4a 100644--- a/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN+++ b/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN@@ -4,9 +4,13 @@ %readonly SYSTEM_CC = "/opt/local/bin/gcc-mp-4.7" %readonly SYSTEM_LIBTOOL = "/opt/local/bin/libtool"-readonly SYSTEM_CC = "/usr/bin/gcc"-readonly SYSTEM_LIBTOOL = "/usr/bin/libtool"-readonly SYSTEM_ASM = "/usr/bin/as"++% Darwin.common does a good job of determining these, esp. the assembler.+% Older /usr/bin/as default to x86 and fail for amd64.+%readonly SYSTEM_CC = "/usr/bin/gcc"+%readonly SYSTEM_LIBTOOL = "/usr/bin/libtool"+%readonly SYSTEM_ASM = "/usr/bin/as"+%readonly SYSTEM_ASM = "/usr/bin/as -arch x86_64" readonly TARGET = "AMD64_DARWIN" readonly GNU_PLATFORM = "i686-darwin8" % "cpu-os" string for GNU Does that really not work for people? Why?Are the cc/libtool on $PATH not acceptable? /usr/bin/as does not work on older systems. At least not w/o -arch. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 11:45:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:45:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? Message-ID: set_range appears to fail to set any bitsif requested to set one bit in a set that doesn't fit in machine word. Test case is here:https://github.com/modula3/cm3/tree/master/m3-sys/m3tests/src/p2/p258 Can anyone try this on a very old cm3 version, like circa 3.6/4.1/5.1?I believe I rewrote this code a few years ago... :( Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 12:01:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 10:01:54 +0000 Subject: [M3devel] m3cg right shift signed? In-Reply-To: References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com>, Message-ID: You are right, it isn't new. But I think nothing in the system was using it until the 32bit wchar support came in.I added test p257 that exercises it..so I can make sure m3cc/nt386 agree, and then update C backend to match. - Jay > Subject: Re: [M3devel] m3cg right shift signed? > From: hosking at purdue.edu > Date: Mon, 8 Jun 2015 22:00:22 -0400 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I don?t think this is a new occurrence. Surely signed right shift makes sense to retain the sign bit? > > > On Jun 8, 2015, at 9:44 PM, Jay wrote: > > > > (Not at propercomputer but sending anyway.) > > > > Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? > > > > - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 12:04:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 10:04:44 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? In-Reply-To: References: Message-ID: The bulk of this is commited now -- 4.3/4.5/4.6 removal.gcc-apple and gmp remain, for now. Any objections and I can add them back. We can also trim the frontends and libraries from 4.7 and apple. Thank you, - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 9 Jun 2015 18:30:17 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? Is there any interest in maintaining 1) gcc/m3cg/parse.c support for gcc 4.3/4.5/4.6? It is slightly messy. 1b) gcc 4.2? This is where hypothetical ARM_DARWIN support is, last I checked, years ago. 2) gcc-4.3/4.5/4.6 in-tree? Or remove them to keep checkouts smaller? There was complaint as to our source tree size, and these directories do add a lot of size for little gain. They were historically useful to transition versions. I don't think it was wrong to have the temporary growth. All targets except for ARM_DARWIN default to gcc 4.7. Do people often/ever go back and compare/debug? Or want to retain that ability, with the current ease? Or ok to get the files from history, for that rare-to-never event? If we do maintain the gcc backend, I would likely import as gcc-5.1 etc., and recreate the problem, but it can be a "rolling" problem and fix. But I'm not keen on maintaining the gcc backend anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 11:59:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:59:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? In-Reply-To: References: Message-ID: Nevermind. I fixed it. I regressed it not in the rewrite in 2010 but in a minor cleanup in 2012. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Wed, 10 Jun 2015 09:45:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? set_range appears to fail to set any bitsif requested to set one bit in a set that doesn't fit in machine word. Test case is here:https://github.com/modula3/cm3/tree/master/m3-sys/m3tests/src/p2/p258 Can anyone try this on a very old cm3 version, like circa 3.6/4.1/5.1?I believe I rewrote this code a few years ago... :( Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 23:51:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 21:51:57 +0000 Subject: [M3devel] rpath woes In-Reply-To: <557138A9.3000605@marino.st> References: <55709183.4060804@marino.st>, , <5570947F.2080600@marino.st>, <5570D114.40104@marino.st>, , <557138A9.3000605@marino.st> Message-ID: It appears FreeBSD system ld gained -z origin support in 4.7. This probably maps back to a particular GNU ld change/version. This issue bugs me. I want something simple and portable. There seems to be no perfect solution. Options include: 1 drop support for old systems 2 relink or chrpath/patchelf upon install (Apple and I think Solaris have similar utilities) 3 better predict the install path at build time I think this is what we do by default actually, just not for building "distributions" (at least make-dist.py) 4 wrapper scripts that set LD_LIBRARY_PATH; seems a popular suggestion I find anything involving wrapper scripts inelegant 5 use libtool and whatever it does, which is probably I'd guess is relink-upon-install 6 use autoconf to detect -z origin and fallback to ? $ORIGIN has the downside of possibly not working with setuid/setgid. yes: http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+10.1-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+10.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+8.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+7.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+6.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+5.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+4.7-RELEASE&arch=default&format=html no: http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+4.0-RELEASE&arch=default&format=html - Jay > Date: Fri, 5 Jun 2015 07:50:33 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > > On 6/5/2015 06:16, Jay K wrote: > > The FreeBSD4 users were surprisingly vocal surprisingly recently. > > So I put some work into it. > > I agree it is an old system. > > It's completely unsupported (for 7 years now) and they don't need the > latest M3 right? Old release still work fine, that should be enough > justification to change this on trunk. > > > > > Linux.common and FreeBSD.common look very similar but I agree different > > and if it is working for you, good, I can ignore it? > > I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? > > Was this right for FreeBSD 5 or 6 or 7 or 8? > > yes. I am using the latest binutils (thus latest ld) with M3. The base > ld is ancient and hand maintained (a GPLv3 issue) so I don't know, > Actually, the linux way is opposite: It explicitly defines SYSTEM_LD > rather than use flags for SYSTEM_CC. So rather than flags, we are > talking about the approach. > > Really nobody with a supported FreeBSD release needs to build from > scratch because it exists in ports. If anybody absolutely wants to, > just say "emulates what the port is doing". This is actually easier > said than done because in addition to using the latest binutils, there > are a number of inline substitutions that are done, see post-patch > target here, starting line 78: > > https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup > > line 85 is where newer gcc and newer as is set for m3. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 23:57:53 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 21:57:53 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556F42FF.9000106@marino.st> References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: By "gcc obsolete targets", I meant that latest gcc might not target systems that we can/do.Not that gcc versions are no longer maintained. Maybe not all that interesting. We had surprisingly recent usage on OSF/1 v4.0. I had wanted to get Irix working. And AIXand I had an older system. Anyway, I forgot about this part of it, and we could targetthem with the C backend instead of gcc. Or make current gcc work. The size is better now? - Jay > Date: Wed, 3 Jun 2015 20:10:07 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On 6/3/2015 20:04, Jay K wrote: > > If gcc obsoletes targets we want to keep, we could keep the old > > versions. Not super useful given our usage levels. > > It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not > obsolete by that definition, so all 5 of these are obsolete. > > I would definitely encourage to prune as many of these as possible. I > could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" > based on the base compiler, why are you assuming it must be built by > base compiler? > > By definition, the base compiler is only required to build base. There > are much newer and well maintained versions of gcc in openbsd ports > tree. There's no reason a ports compiler couldn't be used (I assume > this is actually common). > > > Try xz instead of gzip, maybe it halves the size? > > I can't influence github's API. It is what it is. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jun 12 07:21:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 12 Jun 2015 05:21:44 +0000 Subject: [M3devel] mklib disabled Message-ID: > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > release compiler. > 3. mklib is only used on windows targets anyway. > So, omit mklib altogether, except for windows targets. It is true it did not compile with the released non-Windows m3core. I fixed that. It is true it is only used when targeting Windows. However I want this the way it was. In particular, because w/ it disabled, I have some propensity to break Windows-specific aspects of scripts/python, that leaving it enabled reveals. It is pretty cheap. Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 12 15:39:33 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 08:39:33 -0500 Subject: [M3devel] mklib disabled In-Reply-To: References: Message-ID: <557AE115.8000009@lcwb.coop> On 06/12/2015 12:21 AM, Jay K wrote: > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > release compiler. > > 3. mklib is only used on windows targets anyway. > > So, omit mklib altogether, except for windows targets. > > > It is true it did not compile with the released non-Windows m3core. > I fixed that. > > > It is true it is only used when targeting Windows. > > > However I want this the way it was. > In particular, because w/ it disabled, I have some propensity > to break Windows-specific aspects of scripts/python, that leaving > it enabled reveals. It is pretty cheap. > > > Ok? Not sure I understand, but it sounds like you want to put the github repo in a state where it can not be built by the 5-8 release? If so, that is not OK. The github master branch should always be buildable by the most recent release. Otherwise, we are just insultingly unwelcoming to anybody who is not an insider. You are an insider. Just change it back the way you want in your working directory after you do a pull. > - Jay > > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Jun 12 16:17:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 12 Jun 2015 14:17:30 +0000 Subject: [M3devel] mklib disabled In-Reply-To: <557AE115.8000009@lcwb.coop> References: , <557AE115.8000009@lcwb.coop> Message-ID: > > It is true it did not compile with the released non-Windows m3core. > > I fixed that. > > Not sure I understand, but it sounds like you want to put the github repo > in a state where it can not be built by the 5-8 release? No. It compiles with 5.8.6 now. I fixed that. https://github.com/modula3/cm3/commit/78f21828ad80422b467e31f18c779cebc0a580eb fix for bootstrapping from previous release -- copy in everything used from m3core/winnt.i3 - Jay > Date: Fri, 12 Jun 2015 08:39:33 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] mklib disabled > > > > On 06/12/2015 12:21 AM, Jay K wrote: > > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > > release compiler. > > > 3. mklib is only used on windows targets anyway. > > > So, omit mklib altogether, except for windows targets. > > > > > > It is true it did not compile with the released non-Windows m3core. > > I fixed that. > > > > > > It is true it is only used when targeting Windows. > > > > > > However I want this the way it was. > > In particular, because w/ it disabled, I have some propensity > > to break Windows-specific aspects of scripts/python, that leaving > > it enabled reveals. It is pretty cheap. > > > > > > Ok? > > Not sure I understand, but it sounds like you want to put the github repo > in a state where it can not be built by the 5-8 release? If so, that is > not OK. The github master branch should always be buildable by the most recent > release. Otherwise, we are just insultingly unwelcoming to anybody who > is not an insider. > > You are an insider. Just change it back the way you want in your working directory > after you do a pull. > > > > - Jay > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Fri Jun 12 16:51:53 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 12 Jun 2015 16:51:53 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: References: <55774665.8060303@elstel.org>, Message-ID: <557AF209.6080408@elstel.org> Thanks a lot Rodney and Jay; that will certainly help my implementation. So far all *.mc files found on my machine have the following signature: 16_FD,00,01,{00} except a few text - .mc from PM3 which start alltogether with "begin_unit". Rodney, do you believe that I can rely on the 4th byte to be zero as generated by the Modula-3 middle end. - or would anyone be ready to uphold such a guarantee for the future? Anyone here who has applied "od" on an .mc generated by a very recent compiler? - do they start with 16_FD,10,01,?00? Most binary file types would guarantee a header of at least 4 Byte and it should be more straight forward and secure to check 32bit instead of 24bit if possible. Any suggestions? Am 10.06.15 um 02:21 schrieb Rodney M. Bates: > > > On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >> What are *.mc - files? >> They appear in TARGET - directories; >> most of them are just called _m3main.mc but some of them have other >> names. >> >> I ask because I am writing a program which should recognize and clear >> object files. >> It does not seem to be sufficient to check for uppercase directories >> which are located together with an src directory. >> >> Usually files of a specific type start with a 32bit magic; >> however the mc files all have different starting sequences. >> >> Is there still a straight forward way to recognize an .mc file just >> by its binary content? >> > > They will start with either 16_FD 16_00 16_01, produced by older > versions of cm3, > or 16_FD 16_10 16_01, produced by a very > recent head compiler. > Ignore the 4th byte. Am 09.06.15 um 22:14 schrieb Jay K: > ps: > > foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo > foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io > > again, see cm3 -keep, err better yet, cm3 -keep -verbose > You can see it running cm3cg and as and rm. > > > - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 12 19:28:57 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 12:28:57 -0500 Subject: [M3devel] mklib disabled In-Reply-To: References: , <557AE115.8000009@lcwb.coop> Message-ID: <557B16D9.2090909@lcwb.coop> On 06/12/2015 09:17 AM, Jay K wrote: > > > It is true it did not compile with the released non-Windows m3core. > > > I fixed that. > > > > Not sure I understand, but it sounds like you want to put the github repo > > in a state where it can not be built by the 5-8 release? > > > No. It compiles with 5.8.6 now. I fixed that. OK, I have no problem with that. > > https://github.com/modula3/cm3/commit/78f21828ad80422b467e31f18c779cebc0a580eb > fix for bootstrapping from previous release -- copy in everything used from m3core/winnt.i3 > > - Jay > > > > Date: Fri, 12 Jun 2015 08:39:33 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] mklib disabled > > > > > > > > On 06/12/2015 12:21 AM, Jay K wrote: > > > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > > > release compiler. > > > > 3. mklib is only used on windows targets anyway. > > > > So, omit mklib altogether, except for windows targets. > > > > > > > > > It is true it did not compile with the released non-Windows m3core. > > > I fixed that. > > > > > > > > > It is true it is only used when targeting Windows. > > > > > > > > > However I want this the way it was. > > > In particular, because w/ it disabled, I have some propensity > > > to break Windows-specific aspects of scripts/python, that leaving > > > it enabled reveals. It is pretty cheap. > > > > > > > > > Ok? > > > > Not sure I understand, but it sounds like you want to put the github repo > > in a state where it can not be built by the 5-8 release? If so, that is > > not OK. The github master branch should always be buildable by the most recent > > release. Otherwise, we are just insultingly unwelcoming to anybody who > > is not an insider. > > > > You are an insider. Just change it back the way you want in your working directory > > after you do a pull. > > > > > > > - Jay > > > > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 12 20:00:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 13:00:00 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557AF209.6080408@elstel.org> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> Message-ID: <557B1E20.7090102@lcwb.coop> On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: > > Thanks a lot Rodney and Jay; > that will certainly help my implementation. > > So far all *.mc files found on my machine have the > following signature: > 16_FD,00,01,{00} > > except a few text - .mc from PM3 which start > alltogether with "begin_unit". > > Rodney, do you believe that I can rely on the 4th byte > to be zero as generated by the Modula-3 middle end. - > or would anyone be ready to uphold such a guarantee > for the future? > The 4th byte is not really dependable for the future. It never has had a real magic number. The FD,00,01 is a version number on the binary format, so even it is likely to change. The 4th byte zero is a binary opcode for begin_unit, equivalent to the "begin_unit" in the PM3 text version. I think the most reliable long-term way is just to look for file names *.mc and *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an INTERFACE. These can be regenerated from source and will not be needed once a compile is complete, unless you are into vetting/debugging the compiler. So deleting them is quite safe. I suppose we could add a magic number. We already have a front/back end compatibility change between the release and head compilers. I can do this, if there is consensus we should. How would we choose the number? > Anyone here who has applied "od" on an .mc generated > by a very recent compiler? - do they start with > 16_FD,10,01,?00? > > Most binary file types would guarantee a header of at > least 4 Byte and it should be more straight forward and > secure to check 32bit instead of 24bit if possible. > > Any suggestions? > > > Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >> >> >> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>> What are *.mc - files? >>> They appear in TARGET - directories; >>> most of them are just called _m3main.mc but some of them have other names. >>> >>> I ask because I am writing a program which should recognize and clear object files. >>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>> >>> Usually files of a specific type start with a 32bit magic; >>> however the mc files all have different starting sequences. >>> >>> Is there still a straight forward way to recognize an .mc file just by its binary content? >>> >> >> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >> or 16_FD 16_10 16_01, produced by a very recent head compiler. >> Ignore the 4th byte. > > > Am 09.06.15 um 22:14 schrieb Jay K: >> ps: >> >> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >> >> again, see cm3 -keep, err better yet, cm3 -keep -verbose >> You can see it running cm3cg and as and rm. >> >> >> - Jay >> > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Fri Jun 12 20:51:31 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 12 Jun 2015 20:51:31 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557B1E20.7090102@lcwb.coop> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> Message-ID: <557B2A33.4010402@elstel.org> Am 12.06.15 um 20:00 schrieb Rodney M. Bates: > > > On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: >> >> Thanks a lot Rodney and Jay; >> that will certainly help my implementation. >> >> So far all *.mc files found on my machine have the >> following signature: >> 16_FD,00,01,{00} >> >> except a few text - .mc from PM3 which start >> alltogether with "begin_unit". >> >> Rodney, do you believe that I can rely on the 4th byte >> to be zero as generated by the Modula-3 middle end. - >> or would anyone be ready to uphold such a guarantee >> for the future? >> > > The 4th byte is not really dependable for the future. It never has had > a real magic number. The FD,00,01 is a version number on the binary > format, so even it is likely to change. > > The 4th byte zero is a binary opcode for begin_unit, equivalent > to the "begin_unit" in the PM3 text version. Well, the begin_unit is exactly what I check for when an .mc appears to be text. If 00 encodes begin_unit I believe it should be save to check for FD,00,01,00 and FD,10,01,00. How could an .mc file not start with begin_unit? Wouldn`t that be invalid? - or if it still would be valid I believe we didn`t generate such files, yet. - so if for the future it may start with any other command a fixed 4-byte magic which is not already interpreted would be welcome. Basically any random number should suffice as with 1.000.000 already registered file formats the probability for a clash would just be 1/4000. Nonetheless we could double- check against the database of the "file" program. Not all files have a completely random magic; f.i. pyc (compiled python files) have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be a valid date. However if we can choose things from scratch I would speak for a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers and timestamps in the following (*). It would be beneficial to have at least a cm3-middleend version number encoded since not every backend can be combined with any middle/front-end. * with a version dependent 2-byte header portion I will need a vaildly set current system date to determine whether it is a .pyc of a future version of python. > > I think the most reliable long-term way is just to look for file names > *.mc and > *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an > INTERFACE. These can be regenerated from source and will not be > needed once a > compile is complete, unless you are into vetting/debugging the compiler. > So deleting them is quite safe. Not all *.mc belong to Modula-3. I have some *.mc in my home directory which are in a fact MS Visual Studio files. That is why I prefer a combination of the file extension and file header/magic to determine whether a file can be auto- matically deleted. For Modula-3 it is also quite save to look for TARGET directories**. However if we meet a file which does not contain plain human readable text we may always want to determine in some way where the file stems from and what data it may contain. File suffixes can be stripped by accident (f.i. on an iso9660 file system) or intendedly by will. - and perhaps we do not want to look to deep into a binary before determining what it is (f.i. by a file manager). Even the "file"-tool was already reported to have a security vulnerability ... ** that will at best poorly work on a non-Unix system where file names are not case sensitive. > > I suppose we could add a magic number. We already have a front/back end > compatibility change between the release and head compilers. I can do > this, > if there is consensus we should. How would we choose the number? > > >> Anyone here who has applied "od" on an .mc generated >> by a very recent compiler? - do they start with >> 16_FD,10,01,?00? >> >> Most binary file types would guarantee a header of at >> least 4 Byte and it should be more straight forward and >> secure to check 32bit instead of 24bit if possible. >> >> Any suggestions? >> >> >> Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >>> >>> >>> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>>> What are *.mc - files? >>>> They appear in TARGET - directories; >>>> most of them are just called _m3main.mc but some of them have other >>>> names. >>>> >>>> I ask because I am writing a program which should recognize and >>>> clear object files. >>>> It does not seem to be sufficient to check for uppercase >>>> directories which are located together with an src directory. >>>> >>>> Usually files of a specific type start with a 32bit magic; >>>> however the mc files all have different starting sequences. >>>> >>>> Is there still a straight forward way to recognize an .mc file just >>>> by its binary content? >>>> >>> >>> They will start with either 16_FD 16_00 16_01, produced by older >>> versions of cm3, >>> or 16_FD 16_10 16_01, produced by a very >>> recent head compiler. >>> Ignore the 4th byte. >> >> >> Am 09.06.15 um 22:14 schrieb Jay K: >>> ps: >>> >>> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >>> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >>> >>> again, see cm3 -keep, err better yet, cm3 -keep -verbose >>> You can see it running cm3cg and as and rm. >>> >>> >>> - Jay >>> >> > From jay.krell at cornell.edu Fri Jun 12 22:10:32 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 12 Jun 2015 13:10:32 -0700 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557AF209.6080408@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> Message-ID: <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> No guarantees on any of this imho. Nor the extension. The files are usually temporary. What are the magic bytes for .c? What is the purpose here? We could add 4 ignored bytes or even a guid but it'd be a waste. - Jay On Jun 12, 2015, at 7:51 AM, Elmar Stellnberger wrote: > > Thanks a lot Rodney and Jay; > that will certainly help my implementation. > > So far all *.mc files found on my machine have the > following signature: > 16_FD,00,01,{00} > > except a few text - .mc from PM3 which start > alltogether with "begin_unit". > > Rodney, do you believe that I can rely on the 4th byte > to be zero as generated by the Modula-3 middle end. - > or would anyone be ready to uphold such a guarantee > for the future? > > Anyone here who has applied "od" on an .mc generated > by a very recent compiler? - do they start with > 16_FD,10,01,?00? > > Most binary file types would guarantee a header of at > least 4 Byte and it should be more straight forward and > secure to check 32bit instead of 24bit if possible. > > Any suggestions? > > > Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >> >> >> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>> What are *.mc - files? >>> They appear in TARGET - directories; >>> most of them are just called _m3main.mc but some of them have other names. >>> >>> I ask because I am writing a program which should recognize and clear object files. >>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>> >>> Usually files of a specific type start with a 32bit magic; >>> however the mc files all have different starting sequences. >>> >>> Is there still a straight forward way to recognize an .mc file just by its binary content? >> >> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >> or 16_FD 16_10 16_01, produced by a very recent head compiler. >> Ignore the 4th byte. > > > Am 09.06.15 um 22:14 schrieb Jay K: >> ps: >> >> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >> >> again, see cm3 -keep, err better yet, cm3 -keep -verbose >> You can see it running cm3cg and as and rm. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sat Jun 13 01:07:58 2015 From: adacore at marino.st (John Marino) Date: Sat, 13 Jun 2015 01:07:58 +0200 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: <557B664E.4040801@marino.st> On 6/10/2015 23:57, Jay K wrote: > By "gcc obsolete targets", I meant that latest gcc might not target > systems that we can/do. > Not that gcc versions are no longer maintained. > Maybe not all that interesting. > > > We had surprisingly recent usage on OSF/1 v4.0. I had wanted to get Irix > working. And AIX > and I had an older system. > > Anyway, I forgot about this part of it, and we could target > them with the C backend instead of gcc. Or make current gcc work. > > > The size is better now? Yes. It reduced the distribution file from Github from 150Mb to 91Mb, a significant reduction of 59Mb (almost 40%) Thanks, John From estellnb at elstel.org Sat Jun 13 10:40:40 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Sat, 13 Jun 2015 10:40:40 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> Message-ID: <557BEC88.6020608@elstel.org> Am 12.06.15 um 22:10 schrieb Jay: > No guarantees on any of this imho. Nor the extension. The files are > usually temporary. What are the magic bytes for .c? What is the > purpose here? We could add 4 ignored bytes or even a guid but it'd be > a waste. human readable text files do not need any magic but binaries do (though auto-generated text files by a program will need a header, too). A cm3 and middle end version numbers for a compatibilty check if not also a creation timestamp after the magic would be very beneficial, too. It is unprofessional and a bad habit to emit a binary stream without a header and versioning information. Have you never used a ready compiled cm3cg for compiling a new frontend? - There should be a middle end version number to check whether this is prone to fail (and possibly one trying to force application of cm3cg on a newer intermediate code stream. ). -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jun 13 13:37:43 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Jun 2015 07:37:43 -0400 Subject: [M3devel] On magic numbers In-Reply-To: <557B2A33.4010402@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> Message-ID: <20150613113743.GB23865@topoi.pooq.com> On Fri, Jun 12, 2015 at 08:51:31PM +0200, Elmar Stellnberger wrote: > Basically any random > number should suffice as with 1.000.000 already registered file formats the > probability for a clash would just be 1/4000. Nonetheless we could double- > check against the database of the "file" program. For more collision-freeness for the foreseeable future, I'd suggest a 64-bit random number. Even if there were a collision with someone else's 32-bit number, then next 32 bits would likely resolve the issue. It's not too far-fetched to assume that the number of different file formats will continue increasing exponentially even as our world-wide data storage increases. And maybe it's tie that the hash codes we use for data types also increase in length. I've always considered 32 bits a bit too small for this, especially in the days of *huge* program libraries. Maybe a necessary evil as a concession to antiquated linkers, but it could legitimately be made platform-dependent. For backward copatibility, the compiler could just start checking for the magic number. If it's present, skip it. If it's absent, go on as at present. > Not all files have a completely random magic; f.i. pyc (compiled > python files) > have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be > a valid date. However if we can choose things from scratch I would speak for > a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers > and timestamps in the following (*). > It would be beneficial to have at least a cm3-middleend version number > encoded since not every backend can be combined with any middle/front-end. Of course this should still be appended to the 128 (or however many) bits. -- hendrik From hendrik at topoi.pooq.com Sat Jun 13 13:40:59 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Jun 2015 07:40:59 -0400 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557BEC88.6020608@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> <557BEC88.6020608@elstel.org> Message-ID: <20150613114059.GC23865@topoi.pooq.com> On Sat, Jun 13, 2015 at 10:40:40AM +0200, Elmar Stellnberger wrote: > Am 12.06.15 um 22:10 schrieb Jay: > >No guarantees on any of this imho. Nor the extension. The files > >are usually temporary. What are the magic bytes for .c? What is > >the purpose here? We could add 4 ignored bytes or even a guid but > >it'd be a waste. > human readable text files do not need any magic > but binaries do (though auto-generated text files > by a program will need a header, too). > > A cm3 and middle end version numbers for a > compatibilty check if not also a creation timestamp > after the magic would be very beneficial, too. > > It is unprofessional and a bad habit to emit a binary > stream without a header and versioning information. > Have you never used a ready compiled cm3cg for > compiling a new frontend? - There should be a > middle end version number to check whether this > is prone to fail (and possibly one trying to force > application of cm3cg on a newer intermediate code > stream. ). Not to mention the possibility of having a compatibility mode for an old file format, in case it's not too much work. -- hendrik From rodney_bates at lcwb.coop Sat Jun 13 20:23:20 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:23:20 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557B2A33.4010402@elstel.org> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> Message-ID: <557C7518.6000700@lcwb.coop> On 06/12/2015 01:51 PM, Elmar Stellnberger wrote: > Am 12.06.15 um 20:00 schrieb Rodney M. Bates: >> >> >> On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: >>> >>> Thanks a lot Rodney and Jay; >>> that will certainly help my implementation. >>> >>> So far all *.mc files found on my machine have the >>> following signature: >>> 16_FD,00,01,{00} >>> >>> except a few text - .mc from PM3 which start >>> alltogether with "begin_unit". >>> >>> Rodney, do you believe that I can rely on the 4th byte >>> to be zero as generated by the Modula-3 middle end. - >>> or would anyone be ready to uphold such a guarantee >>> for the future? >>> I doubt anyone would want to guarantee any of these 4 bytes, as they are actual content information and not a magic number. However, I would say the 1st (FD) and last (00, code for begin_unit) have pretty low probability of changing. The middle two contain a version number, so can be expected to change. The second (counting from one) is least significant, and more likely to change. We already have two possible values here, as I reported. >> >> The 4th byte is not really dependable for the future. It never has had >> a real magic number. The FD,00,01 is a version number on the binary >> format, so even it is likely to change. >> >> The 4th byte zero is a binary opcode for begin_unit, equivalent >> to the "begin_unit" in the PM3 text version. > Well, the begin_unit is exactly what I check for when an .mc appears to be text. > If 00 encodes begin_unit I believe it should be save to check for FD,00,01,00 > and FD,10,01,00. How could an .mc file not start with begin_unit? Wouldn`t > that be invalid? - or if it still would be valid I believe we didn`t generate such > files, yet. > - so if for the future it may start with any other command a fixed 4-byte magic > which is not already interpreted would be welcome. Basically any random > number should suffice as with 1.000.000 already registered file formats the > probability for a clash would just be 1/4000. Nonetheless we could double- > check against the database of the "file" program. > Not all files have a completely random magic; f.i. pyc (compiled python files) > have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be > a valid date. However if we can choose things from scratch I would speak for > a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers > and timestamps in the following (*). > It would be beneficial to have at least a cm3-middleend version number > encoded since not every backend can be combined with any middle/front-end. > > * with a version dependent 2-byte header portion I will need a vaildly set current > system date to determine whether it is a .pyc of a future version of python. > >> >> I think the most reliable long-term way is just to look for file names *.mc and >> *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an >> INTERFACE. These can be regenerated from source and will not be needed once a >> compile is complete, unless you are into vetting/debugging the compiler. >> So deleting them is quite safe. > Not all *.mc belong to Modula-3. I have some *.mc in my home directory which > are in a fact MS Visual Studio files. That is why I prefer a combination of the > file extension and file header/magic to determine whether a file can be auto- > matically deleted. OK, file names are not adequate. > For Modula-3 it is also quite save to look for TARGET directories**. However if we > meet a file which does not contain plain human readable text we may always > want to determine in some way where the file stems from and what data it may > contain. File suffixes can be stripped by accident (f.i. on an iso9660 file system) > or intendedly by will. - and perhaps we do not want to look to deep into a > binary before determining what it is (f.i. by a file manager). Even the "file"-tool > was already reported to have a security vulnerability ... > > ** that will at best poorly work on a non-Unix system where file names are not > case sensitive. > >> >> I suppose we could add a magic number. We already have a front/back end >> compatibility change between the release and head compilers. I can do this, >> if there is consensus we should. How would we choose the number? >> >> >>> Anyone here who has applied "od" on an .mc generated >>> by a very recent compiler? - do they start with >>> 16_FD,10,01,?00? >>> >>> Most binary file types would guarantee a header of at >>> least 4 Byte and it should be more straight forward and >>> secure to check 32bit instead of 24bit if possible. >>> >>> Any suggestions? >>> >>> >>> Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >>>> >>>> >>>> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>>>> What are *.mc - files? >>>>> They appear in TARGET - directories; >>>>> most of them are just called _m3main.mc but some of them have other names. >>>>> >>>>> I ask because I am writing a program which should recognize and clear object files. >>>>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>>>> >>>>> Usually files of a specific type start with a 32bit magic; >>>>> however the mc files all have different starting sequences. >>>>> >>>>> Is there still a straight forward way to recognize an .mc file just by its binary content? >>>>> >>>> >>>> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >>>> or 16_FD 16_10 16_01, produced by a very recent head compiler. >>>> Ignore the 4th byte. >>> >>> >>> Am 09.06.15 um 22:14 schrieb Jay K: >>>> ps: >>>> >>>> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >>>> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >>>> >>>> again, see cm3 -keep, err better yet, cm3 -keep -verbose >>>> You can see it running cm3cg and as and rm. >>>> >>>> >>>> - Jay >>>> >>> >> > > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Jun 13 20:27:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:27:00 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557BEC88.6020608@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> <557BEC88.6020608@elstel.org> Message-ID: <557C75F4.4000206@lcwb.coop> On 06/13/2015 03:40 AM, Elmar Stellnberger wrote: > Am 12.06.15 um 22:10 schrieb Jay: >> No guarantees on any of this imho. Nor the extension. The files are usually temporary. What are the magic bytes for .c? What is the purpose here? We could add 4 ignored bytes or even a guid but it'd be a waste. > human readable text files do not need any magic > but binaries do (though auto-generated text files > by a program will need a header, too). > > A cm3 and middle end version numbers for a > compatibilty check if not also a creation timestamp > after the magic would be very beneficial, too. > > It is unprofessional and a bad habit to emit a binary > stream without a header and versioning information. > Have you never used a ready compiled cm3cg for > compiling a new frontend? - There should be a > middle end version number to check whether this > is prone to fail (and possibly one trying to force > application of cm3cg on a newer intermediate code > stream. ). Actually, we have had such a version number, from very early. But it had never changed, until recently, when I changed it to detect this very problem. Now, if you use mismatched cm3cg, you will get an informative error message, instead of something similar to "unrecognized op code" (I paraphrased that.) > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Jun 13 20:42:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:42:26 -0500 Subject: [M3devel] On magic numbers In-Reply-To: <20150613113743.GB23865@topoi.pooq.com> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> <20150613113743.GB23865@topoi.pooq.com> Message-ID: <557C7992.4040103@lcwb.coop> On 06/13/2015 06:37 AM, Hendrik Boom wrote: > On Fri, Jun 12, 2015 at 08:51:31PM +0200, Elmar Stellnberger wrote: > >> Basically any random >> number should suffice as with 1.000.000 already registered file formats the >> probability for a clash would just be 1/4000. Nonetheless we could double- >> check against the database of the "file" program. > > For more collision-freeness for the foreseeable future, I'd suggest a > 64-bit random number. Even if there were a collision with someone > else's 32-bit number, then next 32 bits would likely resolve the issue. > There already is a 64-bit "signature" hash of type structure computed in the compiler, but it is only used in pickles. Elsewhere, a 32-bit "UID" is used. It is just the XOR of the halves of the signature. It was very difficult for me to ferret this conclusion out of the code. The UID is also called something else (I don't remember what) in m3linker and the messages it emits, and may even have more names, making for confusing error messages and difficulty understanding the build system. The fingerprint algorithm has comments suggesting a careful design by someone familiar with high-quality hashes. (That's not me!) Using the full 64 bits everywhere would probably create some annoying transitional compatibility problems. > It's not too far-fetched to assume that the number of different file > formats will continue increasing exponentially even as our world-wide > data storage increases. > > And maybe it's tie that the hash codes we use for data types also > increase in length. I've always considered 32 bits a bit too small for > this, especially in the days of *huge* program libraries. Maybe a > necessary evil as a concession to antiquated linkers, but it could > legitimately be made platform-dependent. > > For backward copatibility, the compiler could just start checking for > the magic number. If it's present, skip it. If it's absent, go on as > at present. > >> Not all files have a completely random magic; f.i. pyc (compiled >> python files) >> have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be >> a valid date. However if we can choose things from scratch I would speak for >> a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers >> and timestamps in the following (*). >> It would be beneficial to have at least a cm3-middleend version number >> encoded since not every backend can be combined with any middle/front-end. > > Of course this should still be appended to the 128 (or however many) > bits. > > -- hendrik > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Jun 14 03:45:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 14 Jun 2015 01:45:19 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <20150613114059.GC23865@topoi.pooq.com> References: <55774665.8060303@elstel.org>, , , <557AF209.6080408@elstel.org>, <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com>, <557BEC88.6020608@elstel.org>, <20150613114059.GC23865@topoi.pooq.com> Message-ID: You should never mix versions. There is no need for a compatibilty mode. You get compatibility mode by building/using an older cm3 and cm3cg together. The system is meant to be tightly coupled. The file format exists I believe primarily as a licensing artifact. When you use the C backend, or the NT386 backend, or presumably LLVM backend, you don't get the files. An approximation of them is used in-memory for the C backend and the NT386 backend never materializes the data, they are just temporary call frames, one at a time. Unless you are in a development mode. Then m3cgcat is a useful dumper of them, and it can translate to C calling into the C backend.. The files are not meant to live long. And even if they did -- hypothetical cm3cginterp, cm3cjit, cm3cgrun -- it'd still be tightly coupled. We are free to change enum values and add/remove fields to each enum. The system has changed through time and there is no need to lock these aspects down. What we need to be compatible with it .m3 and .i3 files. They are currency. There is perhaps some need to be compatible with .o/.mo/.io and even .m3.c/.i3.c. The fact that the C backend has a different ABI than the gcc backend, in particular because of nested functions taking the frame pointer as the last C parameter and not in a special register, worries me, and makes me err toward appending "c" to BUILD_DIR and such, and maybe even TARGET. or "csj" for setjmp -- so that later we have "cppeh" C++ with C++ exception handling.. But anyway, imho .mc files are internal, assuming Tony agrees.. - Jay > Date: Sat, 13 Jun 2015 07:40:59 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3: what are *.mc files > > On Sat, Jun 13, 2015 at 10:40:40AM +0200, Elmar Stellnberger wrote: > > (and possibly one trying to force > > application of cm3cg on a newer intermediate code > > stream. ). > > Not to mention the possibility of having a compatibility mode for an > old file format, in case it's not too much work. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 16 10:43:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 16 Jun 2015 08:43:10 +0000 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu> References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu> Message-ID: I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. From: hosking at cs.purdue.edu Date: Thu, 5 Sep 2013 12:05:07 -0400 To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] TextLiteral.MaxBytes again Here is a modest proposal that should resolve this issue. Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). Updated TextLiteral.i3 and TextLiteral.m3 attached. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 16 10:43:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 16 Jun 2015 08:43:42 +0000 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu>, Message-ID: er..the proposal was old, but nevertheless.. From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: RE: [M3devel] TextLiteral.MaxBytes again Date: Tue, 16 Jun 2015 08:43:10 +0000 I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. From: hosking at cs.purdue.edu Date: Thu, 5 Sep 2013 12:05:07 -0400 To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] TextLiteral.MaxBytes again Here is a modest proposal that should resolve this issue. Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). Updated TextLiteral.i3 and TextLiteral.m3 attached. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 17 18:18:15 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 17 Jun 2015 11:18:15 -0500 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu>, Message-ID: <55819DC7.4090601@lcwb.coop> On 06/16/2015 03:43 AM, Jay K wrote: > er..the proposal was old, but nevertheless.. > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] TextLiteral.MaxBytes again > Date: Tue, 16 Jun 2015 08:43:10 +0000 > > I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. > > If you do this, there are a number of places that should be reviewed for fallout. I think something in runtime has some hard-coded value, and several places there and in pickles use TextLiteral. Plus probably a few tests around. Actually, these should probably be looked at even with the latest change, tho' the latter would not require introduction of unsafe code. > > > From: hosking at cs.purdue.edu > Date: Thu, 5 Sep 2013 12:05:07 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] TextLiteral.MaxBytes again > > Here is a modest proposal that should resolve this issue. > Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; > Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). > > Updated TextLiteral.i3 and TextLiteral.m3 attached. > > > > Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, > tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference > type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 19 03:28:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 18 Jun 2015 20:28:43 -0500 Subject: [M3devel] Anybody using Rd.UnGetCharMulti? Message-ID: <5583704B.1040006@lcwb.coop> Does anybody have code using Rd.UnGetCharMulti? I want to change it a bit to unget several characters in a single call. Some existing calls would have to be changed slightly. -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Mon Jun 22 04:44:20 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 21 Jun 2015 19:44:20 -0700 Subject: [M3devel] ARM_LINUX Bootstrap Compiler Message-ID: Does anyone have a working ARM_LINUX compiler they'd care to post? From the list it does seem some people had it running. If so and you need somewhere to upload it https://github.com/modula3/cm3/releases is probably a good place. - Darko -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Jun 22 10:21:42 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 22 Jun 2015 01:21:42 -0700 Subject: [M3devel] ARM_LINUX Bootstrap Compiler In-Reply-To: References: Message-ID: <20150622082142.E7A8A1A2066@async.async.caltech.edu> I do have it for Raspberry Pi and BeagleBone Black. Works great. But I'll have to look for it. Mika Darko Volaric writes: >--047d7b4140882fa505051912409b >Content-Type: text/plain; charset=UTF-8 > >Does anyone have a working ARM_LINUX compiler they'd care to post? From the >list it does seem some people had it running. > >If so and you need somewhere to upload it >https://github.com/modula3/cm3/releases is probably a good place. > >- Darko > >--047d7b4140882fa505051912409b >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Does anyone have a working=C2=A0ARM_LINUX compiler they= >9;d care to post? From the list it does seem some people had it running.v>
If so and you need somewhere to upload it =C2=A0"https://github.com/modula3/cm3/releases">https://github.com/modula3/cm3/re= >leases =C2=A0is probably a good place.

- Darko= >

> >--047d7b4140882fa505051912409b-- From jay.krell at cornell.edu Mon Jun 22 18:07:55 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 22 Jun 2015 16:07:55 +0000 Subject: [M3devel] ARM_LINUX Bootstrap Compiler In-Reply-To: <20150622082142.E7A8A1A2066@async.async.caltech.edu> References: , <20150622082142.E7A8A1A2066@async.async.caltech.edu> Message-ID: Please try the C backend? Like this: Make sure *target* system has working cc/as/ld/make. cd cm3/scripts/python ./boot1.py ARM_LINUX c You will get something like: cm3-boot-ARM_LINUX-d5.10.0-20150622.tar.gz That works for me. The tail will be replaced by time/date. Copy that to the target, extract, cd, make. See if ./cm3 then works. It should print an error, like: -- building in I386_DARWIN --- Fatal Error: duplicate unit: /cm3/pkg/m3core/src/C/Common/CerrnoC.c ../CerrnoC.c and then, like, have Python installed, and: mkdir /usr/local/cm3/bin mv cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH scripts/python/boot2.sh boot2.sh builds everything. If target system doesn't have cc/as/ld/make, then you cantry using cross tools. Alter $PATH or edit the top of the generated Makefile. And then you can use make-dist.py to build (again) and package everythingand upload that? Thanks, - Jay > To: lists at darko.org > Date: Mon, 22 Jun 2015 01:21:42 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] ARM_LINUX Bootstrap Compiler > > I do have it for Raspberry Pi and BeagleBone Black. Works great. > > But I'll have to look for it. > > Mika > > Darko Volaric writes: > >--047d7b4140882fa505051912409b > >Content-Type: text/plain; charset=UTF-8 > > > >Does anyone have a working ARM_LINUX compiler they'd care to post? From the > >list it does seem some people had it running. > > > >If so and you need somewhere to upload it > >https://github.com/modula3/cm3/releases is probably a good place. > > > >- Darko > > > >--047d7b4140882fa505051912409b > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Does anyone have a working=C2=A0ARM_LINUX compiler they= > >9;d care to post? From the list it does seem some people had it running. >v>
If so and you need somewhere to upload it =C2=A0 >"https://github.com/modula3/cm3/releases">https://github.com/modula3/cm3/re= > >leases =C2=A0is probably a good place.

- Darko= > >

> > > >--047d7b4140882fa505051912409b-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 23 03:50:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 23 Jun 2015 01:50:21 +0000 Subject: [M3devel] pthread assertion failure exiting formsedit Message-ID: Every time I exit formsedit on MacOS X: (gdb) runStarting program: /cm3/bin/formsedit Reading symbols for shared libraries +++++++++++++++++++++..... done ****** runtime error:*** <*ASSERT*> failed.*** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216*** Program received signal SIGABRT, Aborted.0x940181ae in semaphore_wait_signal_trap ()(gdb) bt#0 0x940181ae in semaphore_wait_signal_trap ()#1 0x9404a1c6 in _pthread_cond_wait ()#2 0x9408f449 in pthread_cond_wait ()#3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459#4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239#5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581#6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593#7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52#8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62#9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408#10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115#11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; LOOP IF alertable AND self.alerted THEN self.alerted := FALSE; <*ASSERT self.waitingOn = c.mutex*> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Jun 23 10:41:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 23 Jun 2015 01:41:17 -0700 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: References: Message-ID: <20150623084117.468E41A2069@async.async.caltech.edu> Has anyone run the thread tester with the arguments I specified a few weeks ago on this OS/compiler version? Jay K writes: >--_cbc4bf8e-1039-413b-b25f-4e44135e099e_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Every time I exit formsedit on MacOS X: > >(gdb) runStarting program: /cm3/bin/formsedit Reading symbols for shared li= >braries +++++++++++++++++++++..... done > >****** runtime error:*** <*ASSERT*> failed.*** file "../src/thread/PT= >HREAD/ThreadPThread.m3"=2C line 216*** From rodney_bates at lcwb.coop Fri Jun 26 18:03:53 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 26 Jun 2015 11:03:53 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: References: Message-ID: <558D77E9.9040403@lcwb.coop> This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed on this target. I am rerunning it, but it has gone all night and is still going. self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to top of stack--confusing, to me) looks irrelevant. *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 *** Program received signal SIGABRT, Aborted. [Switching to Thread 140370469975808 (LWP 17456)] 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) up #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) up #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) Current language: auto; currently c (m3gdb) up #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 20 Cstdlib.abort (); Current language: auto; currently Modula-3 (m3gdb) bt #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 #7 0x00007faaa4bad577 in Crash (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) at ../src/runtime/common/RTException.m3:79 #8 0x00007faaa4bad26f in DefaultBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 #9 0x00007faaa4bad1bf in InvokeBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 #10 0x00007faaa4bbad0a in Raise (act= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 #11 0x00007faaa4bad31c in DefaultBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 #12 0x00007faaa4bad1bf in InvokeBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 #13 0x00007faaa4bbad0a in Raise (act= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) at ../src/thread/PTHREAD/ThreadPThread.m3:216 #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) frame 16 #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) at ../src/thread/PTHREAD/ThreadPThread.m3:216 216 <*ASSERT self.waitingOn = c.mutex*> (m3gdb) info arg self = 16_0000000001856130 m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END alertable = TRUE (m3gdb) info loc prev = 16_00007faa8bffeb90 next = 16_00007faaa4bc1db6 (m3gdb) p self^ $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; END; END; END (m3gdb) p c.name $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 (m3gdb) p self.waitingOn $15 = NIL (m3gdb) p c.mutex $16 = 16_00007faa84000e20 (m3gdb) On a previous run, after more poking around in the debugger, I had: (m3gdb) p c.name $12 = (*16_0000000000615b90*) "all editors closed" on 06/22/2015 08:50 PM, Jay K wrote: > Every time I exit formsedit on MacOS X: > > > (gdb) run > Starting program: /cm3/bin/formsedit > Reading symbols for shared libraries +++++++++++++++++++++..... done > > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 > *** > > > Program received signal SIGABRT, Aborted. > 0x940181ae in semaphore_wait_signal_trap () > (gdb) bt > #0 0x940181ae in semaphore_wait_signal_trap () > #1 0x9404a1c6 in _pthread_cond_wait () > #2 0x9408f449 in pthread_cond_wait () > #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 > #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 > #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 > #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 > #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 > #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 > #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 > #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 > #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 > > > WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; > LOOP > IF alertable AND self.alerted THEN > self.alerted := FALSE; > <*ASSERT self.waitingOn = c.mutex*> > > NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. > > > - Jay > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 04:20:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 27 Jun 2015 21:20:34 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: <558D77E9.9040403@lcwb.coop> References: <558D77E9.9040403@lcwb.coop> Message-ID: <558F59F2.8060406@lcwb.coop> Tony, please review/comment on my analysis, as this kind if code is very delicate, and this is the first time I have looked at it. This is in ThreadPThread.m3. It is possible that thread th, waiting on an M3 Condition variable in XWait, line 239, by actually waiting on pthreads condition variable self.cond, can be both Signaled and Alerted, before it reacquires self.mutex. Either of Signal or Alert will pthread_cond_signal self.cond, but the other can acquire self.mutex before th reacquires it, after the first pthread_cond_signal. When th finally gets self.mutex, both self.alerted and self.waitingOn=NIL. I propose to wrap lines 216 thru' 227 inside IF self.waitingOn # NIL THEN ... Presumably, if both Signal and Alert have happened, we want to raise Alerted, rather than just wake up th. On 06/26/2015 11:03 AM, Rodney M. Bates wrote: > This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed > on this target. I am rerunning it, but it has gone all night and is still going. > > self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be > diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to > top of stack--confusing, to me) looks irrelevant. > > > > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 > *** > > > Program received signal SIGABRT, Aborted. > [Switching to Thread 140370469975808 (LWP 17456)] > 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) up > #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) up > #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 > 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) > Current language: auto; currently c > (m3gdb) up > #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 > 20 Cstdlib.abort (); > Current language: auto; currently Modula-3 > (m3gdb) bt > #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 > #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 > #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 > #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 > #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, > msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 > #7 0x00007faaa4bad577 in Crash (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) > at ../src/runtime/common/RTException.m3:79 > #8 0x00007faaa4bad26f in DefaultBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 > #9 0x00007faaa4bad1bf in InvokeBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 > #10 0x00007faaa4bbad0a in Raise (act= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 > #11 0x00007faaa4bad31c in DefaultBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 > #12 0x00007faaa4bad1bf in InvokeBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 > #13 0x00007faaa4bbad0a in Raise (act= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 > #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 > #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) > at ../src/thread/PTHREAD/ThreadPThread.m3:216 > #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 > #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 > #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 > #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 > #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 > #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) frame 16 > #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) > at ../src/thread/PTHREAD/ThreadPThread.m3:216 > 216 <*ASSERT self.waitingOn = c.mutex*> > (m3gdb) info arg > self = 16_0000000001856130 > m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END > c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END > alertable = TRUE > (m3gdb) info loc > prev = 16_00007faa8bffeb90 > next = 16_00007faaa4bc1db6 > (m3gdb) p self^ > $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; > nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; > stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, > Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; > pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; > END; END; END > (m3gdb) p c.name > $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 > (m3gdb) p self.waitingOn > $15 = NIL > (m3gdb) p c.mutex > $16 = 16_00007faa84000e20 > (m3gdb) > > > On a previous run, after more poking around in the debugger, I had: > > (m3gdb) p c.name > $12 = (*16_0000000000615b90*) "all editors closed" > > on 06/22/2015 08:50 PM, Jay K wrote: > > >> Every time I exit formsedit on MacOS X: >> >> >> (gdb) run >> Starting program: /cm3/bin/formsedit >> Reading symbols for shared libraries +++++++++++++++++++++..... done >> >> >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >> *** >> >> >> Program received signal SIGABRT, Aborted. >> 0x940181ae in semaphore_wait_signal_trap () >> (gdb) bt >> #0 0x940181ae in semaphore_wait_signal_trap () >> #1 0x9404a1c6 in _pthread_cond_wait () >> #2 0x9408f449 in pthread_cond_wait () >> #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 >> #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 >> #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 >> #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 >> #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 >> #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 >> #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 >> #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 >> #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 >> >> >> WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; >> LOOP >> IF alertable AND self.alerted THEN >> self.alerted := FALSE; >> <*ASSERT self.waitingOn = c.mutex*> >> >> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. >> >> >> - Jay >> > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 04:29:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 27 Jun 2015 21:29:41 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: <558F59F2.8060406@lcwb.coop> References: <558D77E9.9040403@lcwb.coop> <558F59F2.8060406@lcwb.coop> Message-ID: <558F5C15.4050303@lcwb.coop> I tried this locally (including lines 228 & 229 also) and it does make the current symptom go away. Also, the thread test is not showing any worrisome output, after a few minutes. BTW, I take it this test is not intended to terminate on its own? On 06/27/2015 09:20 PM, Rodney M. Bates wrote: > Tony, please review/comment on my analysis, as this kind if code is very delicate, > and this is the first time I have looked at it. > > This is in ThreadPThread.m3. > > It is possible that thread th, waiting on an M3 Condition variable in XWait, > line 239, by actually waiting on pthreads condition variable self.cond, > can be both Signaled and Alerted, before it reacquires self.mutex. Either of > Signal or Alert will pthread_cond_signal self.cond, but the other can acquire > self.mutex before th reacquires it, after the first pthread_cond_signal. > When th finally gets self.mutex, both self.alerted and self.waitingOn=NIL. > > I propose to wrap lines 216 thru' 227 inside IF self.waitingOn # NIL THEN ... > > Presumably, if both Signal and Alert have happened, we want to raise Alerted, > rather than just wake up th. > > On 06/26/2015 11:03 AM, Rodney M. Bates wrote: >> This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed >> on this target. I am rerunning it, but it has gone all night and is still going. >> >> self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be >> diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to >> top of stack--confusing, to me) looks irrelevant. >> >> >> >> >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >> *** >> >> >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 140370469975808 (LWP 17456)] >> 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) up >> #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) up >> #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 >> 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) >> Current language: auto; currently c >> (m3gdb) up >> #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 >> 20 Cstdlib.abort (); >> Current language: auto; currently Modula-3 >> (m3gdb) bt >> #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 >> #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 >> #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 >> #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 >> #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, >> msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 >> #7 0x00007faaa4bad577 in Crash (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) >> at ../src/runtime/common/RTException.m3:79 >> #8 0x00007faaa4bad26f in DefaultBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 >> #9 0x00007faaa4bad1bf in InvokeBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 >> #10 0x00007faaa4bbad0a in Raise (act= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 >> #11 0x00007faaa4bad31c in DefaultBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 >> #12 0x00007faaa4bad1bf in InvokeBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 >> #13 0x00007faaa4bbad0a in Raise (act= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 >> #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 >> #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) >> at ../src/thread/PTHREAD/ThreadPThread.m3:216 >> #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 >> #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 >> #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 >> #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 >> #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 >> #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) frame 16 >> #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) >> at ../src/thread/PTHREAD/ThreadPThread.m3:216 >> 216 <*ASSERT self.waitingOn = c.mutex*> >> (m3gdb) info arg >> self = 16_0000000001856130 >> m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END >> c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END >> alertable = TRUE >> (m3gdb) info loc >> prev = 16_00007faa8bffeb90 >> next = 16_00007faaa4bc1db6 >> (m3gdb) p self^ >> $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; >> nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; >> stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, >> Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; >> pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; >> END; END; END >> (m3gdb) p c.name >> $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 >> (m3gdb) p self.waitingOn >> $15 = NIL >> (m3gdb) p c.mutex >> $16 = 16_00007faa84000e20 >> (m3gdb) >> >> >> On a previous run, after more poking around in the debugger, I had: >> >> (m3gdb) p c.name >> $12 = (*16_0000000000615b90*) "all editors closed" >> >> on 06/22/2015 08:50 PM, Jay K wrote: >> >> >>> Every time I exit formsedit on MacOS X: >>> >>> >>> (gdb) run >>> Starting program: /cm3/bin/formsedit >>> Reading symbols for shared libraries +++++++++++++++++++++..... done >>> >>> >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >>> *** >>> >>> >>> Program received signal SIGABRT, Aborted. >>> 0x940181ae in semaphore_wait_signal_trap () >>> (gdb) bt >>> #0 0x940181ae in semaphore_wait_signal_trap () >>> #1 0x9404a1c6 in _pthread_cond_wait () >>> #2 0x9408f449 in pthread_cond_wait () >>> #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 >>> #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 >>> #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 >>> #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 >>> #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 >>> #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 >>> #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 >>> #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 >>> #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 >>> >>> >>> WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; >>> LOOP >>> IF alertable AND self.alerted THEN >>> self.alerted := FALSE; >>> <*ASSERT self.waitingOn = c.mutex*> >>> >>> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. >>> >>> >>> - Jay >>> >> > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 19:58:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 28 Jun 2015 12:58:51 -0500 Subject: [M3devel] pthread scheduling fairness of Signal Message-ID: <559035DB.3040101@lcwb.coop> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really what we want? -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 29 15:40:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 29 Jun 2015 08:40:48 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: References: <559035DB.3040101@lcwb.coop> Message-ID: <55914AE0.8080407@lcwb.coop> On 06/28/2015 01:16 PM, Antony Hosking wrote: > It mirrors the user-level threads information as far as I recall. > Do you think that was intentional in the original user-level implementation? It seems like an open invitation to starvation. And it seems even more important for condition variables than mutexen. (hush, spell checker!) Mika's high-volume thread tester continuously churns out "Thread appears starved or deadlocked" messages by the hundreds. > Sent from my iPad > >> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: >> >> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I >> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those >> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really >> what we want? >> -- >> Rodney Bates >> rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From mika at async.caltech.edu Mon Jun 29 17:31:44 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 29 Jun 2015 08:31:44 -0700 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <55914AE0.8080407@lcwb.coop> References: <559035DB.3040101@lcwb.coop> <55914AE0.8080407@lcwb.coop> Message-ID: <20150629153144.B49851A2062@async.async.caltech.edu> Well code shouldn't depend on fairness. I don't think it's a major issue if it's backwards. I didn't add the warning messages about starved threads to the thread tester. "Rodney M. Bates" writes: > > >On 06/28/2015 01:16 PM, Antony Hosking wrote: >> It mirrors the user-level threads information as far as I recall. >> > >Do you think that was intentional in the original user-level implementation? >It seems like an open invitation to starvation. And it seems even more >important for condition variables than mutexen. (hush, spell checker!) > >Mika's high-volume thread tester continuously churns out >"Thread appears starved or deadlocked" messages by the hundreds. > >> Sent from my iPad >> >>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote >: >>> >>> In vetting ThreadPThread in conjunction with the formsedit assertion failur >e, I >>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but >those >>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this r >eally >>> what we want? >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >> > >-- >Rodney Bates >rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 30 02:18:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 29 Jun 2015 19:18:41 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu> References: <559035DB.3040101@lcwb.coop> <55914AE0.8080407@lcwb.coop> <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu> Message-ID: <5591E061.8090400@lcwb.coop> On 06/29/2015 08:53 AM, Antony Hosking wrote: > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: >> >> >> >> On 06/28/2015 01:16 PM, Antony Hosking wrote: >>> It mirrors the user-level threads information as far as I recall. >>> >> >> Do you think that was intentional in the original user-level implementation? >> It seems like an open invitation to starvation. And it seems even more >> important for condition variables than mutexen. (hush, spell checker!) > > Perhaps take a look at the original user-level implementation to confirm. > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening Mutex waiters and LIFO for Condition waiters. Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. (n=# of waiting threads.) I think n will most often be quite small, but occasionally, maybe not, and if/when high thread counts do occur, it would be a good time for things to be fast. It would be quite easy to make both cases FIFO and O(1). No synchronization changes, just change a few lines of code while holding the same locks that are already being held. If nobody objects, I think I will try this. >> >> Mika's high-volume thread tester continuously churns out >> "Thread appears starved or deadlocked" messages by the hundreds. >> >>> Sent from my iPad >>> >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: >>>> >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really >>>> what we want? >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 30 08:37:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 30 Jun 2015 06:37:50 +0000 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <5591E061.8090400@lcwb.coop> References: <559035DB.3040101@lcwb.coop>, , <55914AE0.8080407@lcwb.coop>, <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu>, <5591E061.8090400@lcwb.coop> Message-ID: 1) This sounds good. 2) Can you please apply your analysis to the Win32 version also? Tangentially, I've said both of these before: 3) I'd really like to layer the pthread and Win32 versions over a common interface. 4) Possibly make it much thinner, if Modula-3 semantics are close enough to pthread/win32+cv. - Jay > Date: Mon, 29 Jun 2015 19:18:41 -0500 > From: rodney_bates at lcwb.coop > To: hosking at purdue.edu; mika at async.caltech.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] pthread scheduling fairness of Signal > > > > On 06/29/2015 08:53 AM, Antony Hosking wrote: > > > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: > >> > >> > >> > >> On 06/28/2015 01:16 PM, Antony Hosking wrote: > >>> It mirrors the user-level threads information as far as I recall. > >>> > >> > >> Do you think that was intentional in the original user-level implementation? > >> It seems like an open invitation to starvation. And it seems even more > >> important for condition variables than mutexen. (hush, spell checker!) > > > > Perhaps take a look at the original user-level implementation to confirm. > > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > > > > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening > Mutex waiters and LIFO for Condition waiters. > > Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. > (n=# of waiting threads.) I think n will most often be quite small, but occasionally, > maybe not, and if/when high thread counts do occur, it would be a good time for things > to be fast. > > It would be quite easy to make both cases FIFO and O(1). No synchronization > changes, just change a few lines of code while holding the same locks that are already > being held. > > If nobody objects, I think I will try this. > > >> > >> Mika's high-volume thread tester continuously churns out > >> "Thread appears starved or deadlocked" messages by the hundreds. > >> > >>> Sent from my iPad > >>> > >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: > >>>> > >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I > >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those > >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really > >>>> what we want? > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 30 17:04:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 30 Jun 2015 10:04:12 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: References: <559035DB.3040101@lcwb.coop>, , <55914AE0.8080407@lcwb.coop>, <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu>, <5591E061.8090400@lcwb.coop> Message-ID: <5592AFEC.70906@lcwb.coop> On 06/30/2015 01:37 AM, Jay K wrote: > 1) This sounds good. > 2) Can you please apply your analysis to the Win32 version also? > It looks to me like these matters are entirely delegated to windows-supplied library routines EnterCriticalSection, LeaveCriticalSection, SetEvent, and WaitForMultipleObjects. > > Tangentially, I've said both of these before: > > > 3) I'd really like to layer the pthread and Win32 > versions over a common interface. > > > 4) Possibly make it much thinner, if Modula-3 semantics > are close enough to pthread/win32+cv. > > > - Jay > > > > > Date: Mon, 29 Jun 2015 19:18:41 -0500 > > From: rodney_bates at lcwb.coop > > To: hosking at purdue.edu; mika at async.caltech.edu; m3devel at elegosoft.com > > Subject: Re: [M3devel] pthread scheduling fairness of Signal > > > > > > > > On 06/29/2015 08:53 AM, Antony Hosking wrote: > > > > > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: > > >> > > >> > > >> > > >> On 06/28/2015 01:16 PM, Antony Hosking wrote: > > >>> It mirrors the user-level threads information as far as I recall. > > >>> > > >> > > >> Do you think that was intentional in the original user-level implementation? > > >> It seems like an open invitation to starvation. And it seems even more > > >> important for condition variables than mutexen. (hush, spell checker!) > > > > > > Perhaps take a look at the original user-level implementation to confirm. > > > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > > > > > > > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening > > Mutex waiters and LIFO for Condition waiters. > > > > Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. > > (n=# of waiting threads.) I think n will most often be quite small, but occasionally, > > maybe not, and if/when high thread counts do occur, it would be a good time for things > > to be fast. > > > > It would be quite easy to make both cases FIFO and O(1). No synchronization > > changes, just change a few lines of code while holding the same locks that are already > > being held. > > > > If nobody objects, I think I will try this. > > > > >> > > >> Mika's high-volume thread tester continuously churns out > > >> "Thread appears starved or deadlocked" messages by the hundreds. > > >> > > >>> Sent from my iPad > > >>> > > >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: > > >>>> > > >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I > > >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those > > >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really > > >>>> what we want? > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> > > >> > > >> -- > > >> Rodney Bates > > >> rodney.m.bates at acm.org > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 00:15:26 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:15:26 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B4BE9.1000902@marino.st> References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. - Jay On May 31, 2015, at 10:59 AM, John Marino wrote: > I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > Should I put it back? From the last response I'm guessing that wasn't > the problem. > > John > > > On 5/31/2015 19:09, Jay K wrote: >> Oh probably the problem is using the backend premature actually. >> Darn, I can't help right now. >> >> - Jay >> >> >> >> ------------------------------------------------------------------------ >> From: jay.krell at cornell.edu >> To: adacore at marino.st; m3devel at elegosoft.com >> Date: Sun, 31 May 2015 16:45:42 +0000 >> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >> (0x100), expected 0x110 >> >> Hm. Strange. I need to read more closely. I see it skipping shipping, >> but my read was it didn't try that. >> So.. with INSTALL_CM3_IN_BIN=yes? >> I need to try this all, and "fix" it to set that. The whole thing in >> m3cc/src/m3makefile is relatively >> new vs. when I was actively using all this. :( >> >> - Jay >> >> >>> Date: Sun, 31 May 2015 16:57:02 +0200 >>> From: adacore at marino.st >>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >> (0x100), expected 0x110 >>> >>> On 5/31/2015 11:53, Jay K wrote: >>>> John, have you tried make-dist.py? >>>> i.e. >> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>> >>>> While it might not be exactly what you need, it should demonstrate the >>>> elements. >>>> >>>> I will try to try this all soon, really, I hope so. >>>> >>>> It starts with a working compiler, does no in-place updates, build into >>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>> It does "min" and "all". >>>> >>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>> >>>> - Jay >>> >>> Hi Jay, >>> I'm getting the same error as always using make-dist script: >>> http://leaf.dragonflybsd.org/~marino/m3d.log >>> >>> Thanks, >>> John From jay.krell at cornell.edu Mon Jun 1 00:47:42 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:47:42 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> Ok,caveat that I haven't tried this, could be more waste of your time,but in scripts/pkginfo.txt try moving the m3cc line down to just after cm3. This way, if the config is looking all around for it, as was mentioned and as we should avoid, it won't find it prematurely. I was never quite sure what order to put this entry, as it has in a sense no edges to or from it in the build dependency graph. But there is the sensitivity as to when to use it, so when to ship it. Shipping after cm3 should avoid using it prematurely. - Jay On May 31, 2015, at 3:15 PM, Jay wrote: > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > - Jay > > On May 31, 2015, at 10:59 AM, John Marino wrote: > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >> Should I put it back? From the last response I'm guessing that wasn't >> the problem. >> >> John >> >> >> On 5/31/2015 19:09, Jay K wrote: >>> Oh probably the problem is using the backend premature actually. >>> Darn, I can't help right now. >>> >>> - Jay >>> >>> >>> >>> ------------------------------------------------------------------------ >>> From: jay.krell at cornell.edu >>> To: adacore at marino.st; m3devel at elegosoft.com >>> Date: Sun, 31 May 2015 16:45:42 +0000 >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>> >>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>> but my read was it didn't try that. >>> So.. with INSTALL_CM3_IN_BIN=yes? >>> I need to try this all, and "fix" it to set that. The whole thing in >>> m3cc/src/m3makefile is relatively >>> new vs. when I was actively using all this. :( >>> >>> - Jay >>> >>> >>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>> From: adacore at marino.st >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>>> >>>> On 5/31/2015 11:53, Jay K wrote: >>>>> John, have you tried make-dist.py? >>>>> i.e. >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>> >>>>> While it might not be exactly what you need, it should demonstrate the >>>>> elements. >>>>> >>>>> I will try to try this all soon, really, I hope so. >>>>> >>>>> It starts with a working compiler, does no in-place updates, build into >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>> It does "min" and "all". >>>>> >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>> >>>>> - Jay >>>> >>>> Hi Jay, >>>> I'm getting the same error as always using make-dist script: >>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>> >>>> Thanks, >>>> John From jay.krell at cornell.edu Mon Jun 1 00:51:25 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:51:25 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> Message-ID: <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> Ps. Update pkginfo.txt and then try make-dist.py - Jay On May 31, 2015, at 3:47 PM, Jay wrote: > Ok,caveat that I haven't tried this, could be more waste of your time,but in scripts/pkginfo.txt try moving the m3cc line down to just after cm3. This way, if the config is looking all around for it, as was mentioned and as we should avoid, it won't find it prematurely. > > > I was never quite sure what order to put this entry, as it has in a sense no edges to or from it in the build dependency graph. But there is the sensitivity as to when to use it, so when to ship it. Shipping after cm3 should avoid using it prematurely. > > - Jay > > On May 31, 2015, at 3:15 PM, Jay wrote: > >> I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. >> >> - Jay >> >> On May 31, 2015, at 10:59 AM, John Marino wrote: >> >>> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >>> Should I put it back? From the last response I'm guessing that wasn't >>> the problem. >>> >>> John >>> >>> >>> On 5/31/2015 19:09, Jay K wrote: >>>> Oh probably the problem is using the backend premature actually. >>>> Darn, I can't help right now. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> From: jay.krell at cornell.edu >>>> To: adacore at marino.st; m3devel at elegosoft.com >>>> Date: Sun, 31 May 2015 16:45:42 +0000 >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>>> (0x100), expected 0x110 >>>> >>>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>>> but my read was it didn't try that. >>>> So.. with INSTALL_CM3_IN_BIN=yes? >>>> I need to try this all, and "fix" it to set that. The whole thing in >>>> m3cc/src/m3makefile is relatively >>>> new vs. when I was actively using all this. :( >>>> >>>> - Jay >>>> >>>> >>>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>>> From: adacore at marino.st >>>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>>> (0x100), expected 0x110 >>>>> >>>>> On 5/31/2015 11:53, Jay K wrote: >>>>>> John, have you tried make-dist.py? >>>>>> i.e. >>>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>>> >>>>>> While it might not be exactly what you need, it should demonstrate the >>>>>> elements. >>>>>> >>>>>> I will try to try this all soon, really, I hope so. >>>>>> >>>>>> It starts with a working compiler, does no in-place updates, build into >>>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>>> It does "min" and "all". >>>>>> >>>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>>> >>>>>> - Jay >>>>> >>>>> Hi Jay, >>>>> I'm getting the same error as always using make-dist script: >>>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>>> >>>>> Thanks, >>>>> John From hendrik at topoi.pooq.com Mon Jun 1 02:29:44 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 31 May 2015 20:29:44 -0400 Subject: [M3devel] User interfaces for Modula 3. Message-ID: <20150601002944.GA10751@topoi.pooq.com> What user interface libraries are available for Modula 3? I know there's Trestle. But is there something like GTK or QT or somethng I can use like cairo to draw really pretty text? Anything that supports lots of Unicode? I don't mind havein to do my own character placement based on font metrics of some sort; not now, bit later, I may have some really weird layout constraints. In case you're wodering about the application: I'm doing preliminary planning for something like a text editor with several windows, one with the raw text, and another with a continuous (as continuous as I have cpu cycles for) display of the results of rather complex calculations on that text (such as proof checking or described graphics). Modula 3 isn't the only candidate for a programming language, just the leading candidate for historical and bootstrapping reasons. The others at the moment are OCAML and some kind of statically typed Scheme. And of course, I may decide that theo whole projecct is infeasible. -- hendrik From rodney_bates at lcwb.coop Mon Jun 1 05:28:13 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 31 May 2015 22:28:13 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: <556BD14D.5050507@lcwb.coop> I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes to fail regardless of the order of building the two. OTOH, if cm3 were built and shipped, then m3cc came immediately after, it should work, because m3cc contains no Modula-3 code, and the new cm3 would not, when building it, run cm3cg. After that, there would be a consistent set. for further compiles. But any package containing M3 code built between cm3 and m3cc would fail. On 05/31/2015 05:15 PM, Jay wrote: > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > - Jay > > On May 31, 2015, at 10:59 AM, John Marino wrote: > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >> Should I put it back? From the last response I'm guessing that wasn't >> the problem. >> >> John >> >> >> On 5/31/2015 19:09, Jay K wrote: >>> Oh probably the problem is using the backend premature actually. >>> Darn, I can't help right now. >>> >>> - Jay >>> >>> >>> >>> ------------------------------------------------------------------------ >>> From: jay.krell at cornell.edu >>> To: adacore at marino.st; m3devel at elegosoft.com >>> Date: Sun, 31 May 2015 16:45:42 +0000 >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>> >>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>> but my read was it didn't try that. >>> So.. with INSTALL_CM3_IN_BIN=yes? >>> I need to try this all, and "fix" it to set that. The whole thing in >>> m3cc/src/m3makefile is relatively >>> new vs. when I was actively using all this. :( >>> >>> - Jay >>> >>> >>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>> From: adacore at marino.st >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>>> >>>> On 5/31/2015 11:53, Jay K wrote: >>>>> John, have you tried make-dist.py? >>>>> i.e. >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>> >>>>> While it might not be exactly what you need, it should demonstrate the >>>>> elements. >>>>> >>>>> I will try to try this all soon, really, I hope so. >>>>> >>>>> It starts with a working compiler, does no in-place updates, build into >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>> It does "min" and "all". >>>>> >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>> >>>>> - Jay >>>> >>>> Hi Jay, >>>> I'm getting the same error as always using make-dist script: >>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>> >>>> Thanks, >>>> John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 05:32:29 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 31 May 2015 22:32:29 -0500 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <20150601002944.GA10751@topoi.pooq.com> References: <20150601002944.GA10751@topoi.pooq.com> Message-ID: <556BD24D.7010902@lcwb.coop> There is a QT binding in cm3/m3-ui/qt. I have no experience with it. On 05/31/2015 07:29 PM, Hendrik Boom wrote: > What user interface libraries are available for Modula 3? > > I know there's Trestle. > > But is there something like GTK or QT or somethng I can use like cairo > to draw really pretty text? Anything that supports lots of Unicode? > I don't mind havein to do my own character placement based on font > metrics of some sort; not now, bit later, I may have some really > weird layout constraints. > > In case you're wodering about the application: > > I'm doing preliminary planning for something like a text editor with > several windows, one with the raw text, and another with a continuous > (as continuous as I have cpu cycles for) display of the results of rather > complex calculations on that text (such as proof checking or > described graphics). > > Modula 3 isn't the only candidate for a programming language, just the > leading candidate for historical and bootstrapping reasons. The others > at the moment are OCAML and some kind of statically typed Scheme. > > And of course, I may decide that theo whole projecct is infeasible. > > -- hendrik > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 08:40:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:40:38 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556BD14D.5050507@lcwb.coop> References: <556700B0.8060500@marino.st> <5567DA8C.3050802@lcwb.coop>,<556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , <556BD14D.5050507@lcwb.coop> Message-ID: Yes, exactly, that's what I was trying to say, in my later/last mail. Yes the compability is definitely two-way. There is a one to one mapping back and forth. cm3 goes with cm3cg; cm3cg goes with cm3. Neither is compatible with older nor necessarily newer versions. They would be statically linked together if not for the GPL. When I first placed m3cc in pkginfo.txt years ago, I was only thinking of build order and, like, "override". m3cc imports nothing and nothing imports m3cc. In the Modula-3 module/library/interface sense. It can be built when you have nothing. So I figured put it nearly first. Only now today did I consider the shipping order. It should be shipped adjacent to cm3, either right before or right after. If you split build and ship, then it can still be built early, or at almost any time. But we have one ordering for build and ship, so let the shipping order decide. I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. I still think it broke upgrade.py. I can workaround. But I'd like to remove the environment variable check. There are many order dependencies in build and ship. This is just one. - Jay > Date: Sun, 31 May 2015 22:28:13 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; adacore at marino.st > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > to fail regardless of the order of building the two. > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > it should work, because m3cc contains no Modula-3 code, and the new > cm3 would not, when building it, run cm3cg. After that, there would > be a consistent set. for further compiles. But any package containing > M3 code built between cm3 and m3cc would fail. > > On 05/31/2015 05:15 PM, Jay wrote: > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > - Jay > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > >> Should I put it back? From the last response I'm guessing that wasn't > >> the problem. > >> > >> John > >> > >> > >> On 5/31/2015 19:09, Jay K wrote: > >>> Oh probably the problem is using the backend premature actually. > >>> Darn, I can't help right now. > >>> > >>> - Jay > >>> > >>> > >>> > >>> ------------------------------------------------------------------------ > >>> From: jay.krell at cornell.edu > >>> To: adacore at marino.st; m3devel at elegosoft.com > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >>> (0x100), expected 0x110 > >>> > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > >>> but my read was it didn't try that. > >>> So.. with INSTALL_CM3_IN_BIN=yes? > >>> I need to try this all, and "fix" it to set that. The whole thing in > >>> m3cc/src/m3makefile is relatively > >>> new vs. when I was actively using all this. :( > >>> > >>> - Jay > >>> > >>> > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > >>>> From: adacore at marino.st > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >>> (0x100), expected 0x110 > >>>> > >>>> On 5/31/2015 11:53, Jay K wrote: > >>>>> John, have you tried make-dist.py? > >>>>> i.e. > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > >>>>> > >>>>> While it might not be exactly what you need, it should demonstrate the > >>>>> elements. > >>>>> > >>>>> I will try to try this all soon, really, I hope so. > >>>>> > >>>>> It starts with a working compiler, does no in-place updates, build into > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > >>>>> It does "min" and "all". > >>>>> > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > >>>>> > >>>>> - Jay > >>>> > >>>> Hi Jay, > >>>> I'm getting the same error as always using make-dist script: > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > >>>> > >>>> Thanks, > >>>> John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 08:54:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:54:29 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B5356.40809@lcwb.coop> References: <556700B0.8060500@marino.st>, <5567DA8C.3050802@lcwb.coop>,,<556817A1.2090206@marino.st>, ,,<20150529105451.e9c9b9c2a5578a28bc654f97@elego.de>, ,,<55682D1A.2030504@marino.st>, ,,<20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de>, ,,<55683689.2090709@marino.st>, ,,<20150529122732.1396eb079ff9b404bcb21e40@elego.de>, ,,<5568421D.3050209@marino.st>, ,,<20150529155833.8454664b88391d93708e8f85@elego.de>, ,,<55687266.3090400@marino.st>, ,,<20150529162028.388911007a614006049beb42@elego.de>, ,,<20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, ,,<556885FD.6010800@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, ,,<55689065.1040207@marino.st>, ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, ,,, , , <556B213E.8030108@marino.st>, , , <556B4BE9.1000902@marino .st>,<556B5356.40809@lcwb.coop> Message-ID: It only happens if you ship. Shipping in the wrong order can always break things. You cannot ship in arbitrary order and incorrect order can cause arbitrary problems. That doesn't mean we should never ship and always have a separate copy mechanism. The knowledge of what to ship is meant to be in the m3makefiles. Not also encoded elsewhere. The environment variable check really should be removed. Perhaps perhaps perh aps shipping order can be enforced? For things other than m3cc, this seems possible. Immediate matching dependencies should be present in the destination. Using the information that is already present in the system as to immediate and matching, in terms of Modula-3 imports/interfaces. m3cc is unique in not fitting in that however. As well, the dependency between m3cc/cm3 -- what order would you say it is? Circular but the tie can be broken arbitrarily? Introduce a command that can ship multiple packages at once? I kind of want that anyway, but in this case you'd arrive back at the environment variable, as long as cm3 has it..though cm3 only needs it for NT and hypothetical other targets like AIX/OS2/DOS/DJGPP. However -- how about this suggestion -- make the environment variable check for relative native target? Only skip cm3 shipping when we know it doesn't work, I..e NT/Cygwin/Mingwin? I like this. It is a special case either way, but at least we can demonstrate a cleaner system on most systems.. - Jay > Date: Sun, 31 May 2015 13:30:46 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; adacore at marino.st > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 12:59 PM, John Marino wrote: > > I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > Should I put it back? From the last response I'm guessing that wasn't > > the problem. > > > > John > > > > It apparently wasn't the only problem, but it definitely was a problem. > Don't put it back. I causes the new cm3cg executable to be copied > to your bin directory immediately after it has been built. It is > incompatible with the old cm3 executable, and the next attempted > build (m3core, as I remember) runs the old cm3, then the new cm3cg, > which correctly recognizes that its input is in an old version format > and refuses, giving the error message. > > > > > On 5/31/2015 19:09, Jay K wrote: > >> Oh probably the problem is using the backend premature actually. > >> Darn, I can't help right now. > >> > >> - Jay > >> > >> > >> > >> ------------------------------------------------------------------------ > >> From: jay.krell at cornell.edu > >> To: adacore at marino.st; m3devel at elegosoft.com > >> Date: Sun, 31 May 2015 16:45:42 +0000 > >> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >> (0x100), expected 0x110 > >> > >> Hm. Strange. I need to read more closely. I see it skipping shipping, > >> but my read was it didn't try that. > >> So.. with INSTALL_CM3_IN_BIN=yes? > >> I need to try this all, and "fix" it to set that. The whole thing in > >> m3cc/src/m3makefile is relatively > >> new vs. when I was actively using all this. :( > >> > >> - Jay > >> > >> > >>> Date: Sun, 31 May 2015 16:57:02 +0200 > >>> From: adacore at marino.st > >>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >> (0x100), expected 0x110 > >>> > >>> On 5/31/2015 11:53, Jay K wrote: > >>>> John, have you tried make-dist.py? > >>>> i.e. > >> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > >>>> > >>>> While it might not be exactly what you need, it should demonstrate the > >>>> elements. > >>>> > >>>> I will try to try this all soon, really, I hope so. > >>>> > >>>> It starts with a working compiler, does no in-place updates, build into > >>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > >>>> It does "min" and "all". > >>>> > >>>> If you set the STAGE environment variable, it uses that instead of /tmp. > >>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > >>>> > >>>> - Jay > >>>> > >>> > >>> Hi Jay, > >>> I'm getting the same error as always using make-dist script: > >>> http://leaf.dragonflybsd.org/~marino/m3d.log > >>> > >>> Thanks, > >>> John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 08:48:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:48:12 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B5D6A.3030403@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B5D6A.3030403@lcwb.coop> Message-ID: > was looking several alternative places for an executable cm3cg I agree it did/does do that. I did put that in. I thought it was a good idea at the time. I realize now it is a mistake. If that behavior is still present, it will be advisable also to first update one's config to stop doing that. Basically, "ship" is a very carefully ordered operation, the shipping of multiple packages. If you have no bugs, and you ship in the correct order, then you don't need DESTDIR. IF you have no bugs, but you ship in the incorrect order, you cause a bad mixing of things. If you use "DESTDIR" then you can ship safely in any order and do some switch at the end, such as by reseting PATH to point to the new set of files. If you have circular dependencies, there might not be a workable order. I agree this could be the case. make-dist.py is written to avoid but might be slightly wrong. I suspect I only ever ran make-dist after upgrade. I have started setting up VMs..though NetBSD, FreeBSD, OpenBSD all fail under VirtualBox if Hyper-V is enabled...but at least Debian is making progress. So hopefully I can test and fix this properly soon. And then I'll back to using the C backend anyway. :) - Jay > Date: Sun, 31 May 2015 14:13:46 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > The error message you keep getting pretty certainly means it is running a > new cm3cg with an old cm3. And it consistently happens immediately after > cm3cg is rebuilt. > > The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > your failed builds, and I don't think that package would ever run cm3cg, as > there is no Modula-3 code in it. Which means there is no proof that the > version of cm3cg that would have been used before it was rebuilt is the old > one. (Could it be a new one left over from a previous incomplete build?) > > One experiment that would prove this would be, in the state your are > starting from before trying a full build, try any means of running > cm3 on a package that contains Modula-3 code, to see if it avoids the > recurring problem. > > But I have another theory. I don't know what this ports bootstrap > is, but there was a time when things where set up so that cm3, instead > of looking in just the same directory as its own executable was found, > was looking several alternative places for an executable cm3cg. This > caused the same problem to occur in a different way. Even when the > newly rebuilt cm3cg executable was not copied to the bin directory > right away, its mere existence where it was built was causing it to > be found the next time cm3 started up, leading to the same failure. > > I don't remember how this was fixed, but either Jay or I did something > about it. Perhaps the ports bootstrap was made during a time when this > behavior was happening. > > On 05/31/2015 02:41 AM, John Marino wrote: > > On 5/29/2015 20:43, John Marino wrote: > >> On 5/29/2015 20:15, Olaf Wagner wrote: > >>> > >>> Well, yes, I understand that. I would have tried your exact setup, > >>> but I have no system available to test that on. > >>> > >>> At least I have validated that based on the origianl 5.8.6 installation > >>> archive for AMD64_FREEBSD you can build the new compiler from the current > >>> sources with a simple call of the upgrade.sh script. which I still doubted > >>> yesterday. > >> > >> > >> The card I still have left to play is to extract the bootstrap, let it > >> overwrite itself per Rodney's technique and then build the real compiler > >> (dumping the whole "intermediate" area). > > > > FWIW, this did not work either. Rodney's technique doesn't seem to work > > with the ports bootstrap. I don't know why it would work with provided > > 5.8.6 but not rebuilt one. As far as I can tell, I did what he said > > would always work. > > > > I'm at a loss as to where to go from here. It's starting to look like I > > have to throw away the ports 5.8.6 bootstrap and start over somehow. I > > don't think it should be this hard. :( > > > > John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 09:05:31 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:05:31 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B743E.7030504@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B5D6A.3030403@lcwb.coop>, <556B6075.6010703@marino.st>, <556B743E.7030504@lcwb.coop> Message-ID: > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > 274: foreach root in [ m3cc, bin ] > to: > 274: foreach root in [ bin ] > Before building the Modula-3 head. Yes, agreed. I still want the environment variable check removed, but this is another problem, that I caused, and I did fix long ago, but is causing grief still, sorry about that. Here, I fixed it in 2010: https://github.com/modula3/cm3/commit/65551f30a6c3ed2cd9740394a9082ffab1b07faa INSTALL_ROOT/bin/cm3cg for native builds ROOT/m3-sys/m3cc/HOST/TARGET/cm3cg for cross builds "Later" (never?) we can think about "installed" cross builds. In particular, when I make m3cg interface changes, I get burned by reaching in for ROOT/m3-sys/m3cc/x/cm3cg. The code is a lot smaller now and more correct. Sorry about that. I was being ambitious in trying to make things work, and it turned out to be quite incorrect once things are better understood. - Jay > Date: Sun, 31 May 2015 15:51:10 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 02:26 PM, John Marino wrote: > > On 5/31/2015 21:13, Rodney M. Bates wrote: > >> The error message you keep getting pretty certainly means it is running a > >> new cm3cg with an old cm3. And it consistently happens immediately after > >> cm3cg is rebuilt. > >> > >> The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > >> your failed builds, and I don't think that package would ever run cm3cg, as > >> there is no Modula-3 code in it. Which means there is no proof that the > >> version of cm3cg that would have been used before it was rebuilt is the old > >> one. (Could it be a new one left over from a previous incomplete build?) > > > > That's not possible. The bootstrap is packaged in a compressed tarball > > and extracted for the sole purpose of building M3. It would have not > > leftovers in it. > > > > It comes with cm3, cm3cg, m3bundle, and mklib in the bin directory. > > > > the contents of the bootstrap M3 cm3.cfg are: > > INSTALL_ROOT = path() & "/.." > > include (path() & "/../etc/modula3/AMD64_FREEBSD") > > > > > >> One experiment that would prove this would be, in the state your are > >> starting from before trying a full build, try any means of running > >> cm3 on a package that contains Modula-3 code, to see if it avoids the > >> recurring problem. > > > > I think this is N/A. The bootstrap compiler is extracted to a directory > > that is not on the path (and only exists right before trying to build > > modula3 port) > > > > > >> But I have another theory. I don't know what this ports bootstrap > >> is, but there was a time when things where set up so that cm3, instead > >> of looking in just the same directory as its own executable was found, > >> was looking several alternative places for an executable cm3cg. This > > > > It's 5.8.6 and the cm3.cfg is listed above. You can download it > > yourself here: > > http://downloads.dragonlace.net/m3/m3-bootstrap.AMD64.FREEBSD.92.tar.bz2 > > > > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > > 274: foreach root in [ m3cc, bin ] > > to: > > 274: foreach root in [ bin ] > > Before building the Modula-3 head. > > > > >> caused the same problem to occur in a different way. Even when the > >> newly rebuilt cm3cg executable was not copied to the bin directory > >> right away, its mere existence where it was built was causing it to > >> be found the next time cm3 started up, leading to the same failure. > >> > >> I don't remember how this was fixed, but either Jay or I did something > >> about it. Perhaps the ports bootstrap was made during a time when this > >> behavior was happening. > > > > Probably not as I am 99% sure the bootstrap was built from the release > > source. > > > > Thanks, > > John > > > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 09:07:25 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 09:07:25 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> References: <556700B0.8060500@marino.st> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> Message-ID: <556C04AD.5000509@marino.st> On 6/1/2015 00:51, Jay wrote: > Ps. Update pkginfo.txt and then try make-dist.py > Okay, I applied this patch: http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt and the same thing happened, only later: http://leaf.dragonflybsd.org/~marino/m3e.log John From jay.krell at cornell.edu Mon Jun 1 09:05:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:05:05 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B743E.7030504@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B5D6A.3030403@lcwb.coop>, <556B6075.6010703@marino.st>, <556B743E.7030504@lcwb.coop> Message-ID: > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > 274: foreach root in [ m3cc, bin ] > to: > 274: foreach root in [ bin ] > Before building the Modula-3 head. Yes, agreed. I still want the environment variable check removed, but this is another problem, that I caused, and I did fix long ago, but is causing grief still, sorry about that. Here, I fixed it in 2010: https://github.com/modula3/cm3/commit/65551f30a6c3ed2cd9740394a9082ffab1b07faa INSTALL_ROOT/bin/cm3cg for native builds ROOT/m3-sys/m3cc/HOST/TARGET/cm3cg for cross builds "Later" (never?) we can think about "installed" cross builds. In particular, when I make m3cg interface changes, I get burned by reaching in for ROOT/m3-sys/m3cc/x/cm3cg. The code is a lot smaller now and more correct. Sorry about that. I was being ambitious in trying to make things work, and it turned out to be quite incorrect once things are better understood. - Jay > Date: Sun, 31 May 2015 15:51:10 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 02:26 PM, John Marino wrote: > > On 5/31/2015 21:13, Rodney M. Bates wrote: > >> The error message you keep getting pretty certainly means it is running a > >> new cm3cg with an old cm3. And it consistently happens immediately after > >> cm3cg is rebuilt. > >> > >> The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > >> your failed builds, and I don't think that package would ever run cm3cg, as > >> there is no Modula-3 code in it. Which means there is no proof that the > >> version of cm3cg that would have been used before it was rebuilt is the old > >> one. (Could it be a new one left over from a previous incomplete build?) > > > > That's not possible. The bootstrap is packaged in a compressed tarball > > and extracted for the sole purpose of building M3. It would have not > > leftovers in it. > > > > It comes with cm3, cm3cg, m3bundle, and mklib in the bin directory. > > > > the contents of the bootstrap M3 cm3.cfg are: > > INSTALL_ROOT = path() & "/.." > > include (path() & "/../etc/modula3/AMD64_FREEBSD") > > > > > >> One experiment that would prove this would be, in the state your are > >> starting from before trying a full build, try any means of running > >> cm3 on a package that contains Modula-3 code, to see if it avoids the > >> recurring problem. > > > > I think this is N/A. The bootstrap compiler is extracted to a directory > > that is not on the path (and only exists right before trying to build > > modula3 port) > > > > > >> But I have another theory. I don't know what this ports bootstrap > >> is, but there was a time when things where set up so that cm3, instead > >> of looking in just the same directory as its own executable was found, > >> was looking several alternative places for an executable cm3cg. This > > > > It's 5.8.6 and the cm3.cfg is listed above. You can download it > > yourself here: > > http://downloads.dragonlace.net/m3/m3-bootstrap.AMD64.FREEBSD.92.tar.bz2 > > > > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > > 274: foreach root in [ m3cc, bin ] > > to: > > 274: foreach root in [ bin ] > > Before building the Modula-3 head. > > > > >> caused the same problem to occur in a different way. Even when the > >> newly rebuilt cm3cg executable was not copied to the bin directory > >> right away, its mere existence where it was built was causing it to > >> be found the next time cm3 started up, leading to the same failure. > >> > >> I don't remember how this was fixed, but either Jay or I did something > >> about it. Perhaps the ports bootstrap was made during a time when this > >> behavior was happening. > > > > Probably not as I am 99% sure the bootstrap was built from the release > > source. > > > > Thanks, > > John > > > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Mon Jun 1 09:11:22 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 1 Jun 2015 09:11:22 +0200 (CEST) Subject: [M3devel] duplicate subscription Message-ID: It seems that I am subscribed twice to this list. How can I find out the according e-mail addresses? From adacore at marino.st Mon Jun 1 09:19:01 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 09:19:01 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C04AD.5000509@marino.st> References: <556700B0.8060500@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> Message-ID: <556C0765.5050607@marino.st> On 6/1/2015 09:07, John Marino wrote: > On 6/1/2015 00:51, Jay wrote: >> Ps. Update pkginfo.txt and then try make-dist.py >> > > Okay, I applied this patch: > http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt > > and the same thing happened, only later: > http://leaf.dragonflybsd.org/~marino/m3e.log > Well, not the "same" thing. ${STAGE}/compiler_with_self/bin seems to be installed. I didn't change bootstrap/etc/modula3/cm3cfg.common though, it sounds like I shouldn't after all. Should I? John From jay.krell at cornell.edu Mon Jun 1 09:35:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:35:19 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C0765.5050607@marino.st> References: <556700B0.8060500@marino.st>, <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st> Message-ID: Given that you seem to be very quick and very patient, then yes, try that. I am trying to get back up to speed in all this, but you are way ahead of me. The actual fix in 2010 was more than Rodney described. It was removing a bunch of code. The theory is that we are still reaching around and finding the wrong cm3cg somewhere, instead of going directly to the matching one in the one place it belongs (ignoring cross build matters). It isn't entirely clear though, like where would it find the wrong one. I understand how it used to go bad . see: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common I hope to get catch up soon.. :( on being away.. - Jay > Date: Mon, 1 Jun 2015 09:19:01 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 09:07, John Marino wrote: > > On 6/1/2015 00:51, Jay wrote: > >> Ps. Update pkginfo.txt and then try make-dist.py > >> > > > > Okay, I applied this patch: > > http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt > > > > and the same thing happened, only later: > > http://leaf.dragonflybsd.org/~marino/m3e.log > > > > > Well, not the "same" thing. > ${STAGE}/compiler_with_self/bin seems to be installed. > > I didn't change bootstrap/etc/modula3/cm3cfg.common though, it sounds > like I shouldn't after all. Should I? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Jun 1 10:02:13 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 1 Jun 2015 10:02:13 +0200 Subject: [M3devel] duplicate subscription In-Reply-To: References: Message-ID: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> On Mon, 1 Jun 2015 09:11:22 +0200 (CEST) Henning Thielemann wrote: > > It seems that I am subscribed twice to this list. How can I find out the > according e-mail addresses? Mailing list information can be found at https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel (or /m3commit, /m3announce). Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From adacore at marino.st Mon Jun 1 10:04:40 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:04:40 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st> Message-ID: <556C1218.3000103@marino.st> On 6/1/2015 09:35, Jay K wrote: > Given that you seem to be very quick and very patient, then yes, try that. > I am trying to get back up to speed in all this, but you are way ahead > of me. After this morning, I'll have to put this away until the weekend I think. However, changing that file in addition almost worked: http://leaf.dragonflybsd.org/~marino/m3f.log Failed at the end of log: "/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib/src/../../quake/cvsup.quake", line 117: quake runtime error: undefined variable: IsDarwin I'm guessing a simple patch to cvsup.quake would be sufficient, although it would me the error is current. John From wagner at elegosoft.com Mon Jun 1 10:13:09 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 1 Jun 2015 10:13:09 +0200 Subject: [M3devel] cm3/m3quake: setting environment variables for exec In-Reply-To: <556B49DD.20909@elstel.org> References: <556B49DD.20909@elstel.org> Message-ID: <20150601101309.2ac490ee3305b476a29be2ec@elegosoft.com> On Sun, 31 May 2015 19:50:21 +0200 Elmar Stellnberger wrote: > How can I set environment variables for exec with cm3? > Using /usr/bin/env would be inherently non-portable. > With SRC M3 I remember this was the 4th parameter or so. > However cm3 exec seems to do something very different. I don't know about that. The complete quake specification can be found at http://www.opencm3.net/doc/help/cm3/quake.html including all built-in functions and procedures. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 1 10:15:35 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 08:15:35 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1218.3000103@marino.st> References: <556700B0.8060500@marino.st>, <20150529122732.1396eb079ff9b404bcb21e40@elego.de>, <5568421D.3050209@marino.st>, <20150529155833.8454664b88391d93708e8f85@elego.de>, <55687266.3090400@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, <556885FD.6010800@marino.st>, <20150529180223.d467621b68ebf665da685b12@elego.de>, <55689065.1040207@marino.st>, <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B213E.8030108@marino.st>, , <556B4BE9.1000902@marino.st>, , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , <556C1218.3000103@marino.st> Message-ID: This is very good progress. Thank you for your speed and patience. And the error is kind of minor and uninteresting. Yes, I understand, it is fatal. Everything should just work to completion. See IsDarwin here: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common cvsup is fiddling around how to use zlib. See here: https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile and surely cvsup isn't interesting any more? Who is using cvs? You can delete m3-tools/cvsup. or change the scripts to skip it or update cm3cfg.common to be more up to date. which brings me to...seems like maybe an incorrect mixing of bootstrap config and latest config? Which begs the question though to which parts of config are local-specific and shouldn't be updated/replaced, and which parts are bound to cm3 and should be replaced. Which really suggest a slight refactoring of the config files. They are nicely factored, but not quite along these lines. Arguably whatever is bound to cm3 should be written in Modula-3 and statically linked, not in these free floating text files. But quake is convenient, and fast enough seemingly... - Jay > Date: Mon, 1 Jun 2015 10:04:40 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 09:35, Jay K wrote: > > Given that you seem to be very quick and very patient, then yes, try that. > > I am trying to get back up to speed in all this, but you are way ahead > > of me. > > After this morning, I'll have to put this away until the weekend I think. > > However, changing that file in addition almost worked: > http://leaf.dragonflybsd.org/~marino/m3f.log > > Failed at the end of log: > "/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib/src/../../quake/cvsup.quake", > line 117: quake runtime error: undefined variable: IsDarwin > > > I'm guessing a simple patch to cvsup.quake would be sufficient, although > it would me the error is current. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 10:25:11 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:25:11 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <5568421D.3050209@marino.st>, <20150529155833.8454664b88391d93708e8f85@elego.de>, <55687266.3090400@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, <556885FD.6010800@marino.st>, <20150529180223.d467621b68ebf665da685b12@elego.de>, <55689065.1040207@marino.st>, <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B213E.8030108@marino.st>, , <556B4BE9.1000902@marino.st>, , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , <556C1218.3000103@marino.st> Message-ID: <556C16E7.505@marino.st> On 6/1/2015 10:15, Jay K wrote: > This is very good progress. > Thank you for your speed and patience. > > > And the error is kind of minor and uninteresting. Yes, I understand, it > is fatal. Everything should just work to completion. > > > See IsDarwin here: > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > cvsup is fiddling around how to use zlib. > See here: > https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile > > > and surely cvsup isn't interesting any more? Who is using cvs? NetBSD and OpenBSD for starters. > You can delete m3-tools/cvsup. > or change the scripts to skip it > or update cm3cfg.common to be more up to date. I'd like to keep it, it's the best source for cvsup. I thought make-dist.py was just going to bootstrap (e.g. minimal required). Is it actually building everything? I was intending to run do-cm3-min.sh buildship, do-cm3-std.sh buildship, do-cm3-caltech-parser.sh buildship after make-dist.py completed. John From dmuysers at hotmail.com Mon Jun 1 10:26:37 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Mon, 1 Jun 2015 10:26:37 +0200 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <20150601002944.GA10751@topoi.pooq.com> References: <20150601002944.GA10751@topoi.pooq.com> Message-ID: Yes, there is libiup. Interface is portable C. Remains to write an M3 interface file. And it is pure Unicode (UTF-8), as it should be everywhere -----Original Message----- From: Hendrik Boom Sent: Monday, June 01, 2015 2:29 AM To: m3devel Subject: [M3devel] User interfaces for Modula 3. What user interface libraries are available for Modula 3? I know there's Trestle. But is there something like GTK or QT or somethng I can use like cairo to draw really pretty text? Anything that supports lots of Unicode? I don't mind havein to do my own character placement based on font metrics of some sort; not now, bit later, I may have some really weird layout constraints. In case you're wodering about the application: I'm doing preliminary planning for something like a text editor with several windows, one with the raw text, and another with a continuous (as continuous as I have cpu cycles for) display of the results of rather complex calculations on that text (such as proof checking or described graphics). Modula 3 isn't the only candidate for a programming language, just the leading candidate for historical and bootstrapping reasons. The others at the moment are OCAML and some kind of statically typed Scheme. And of course, I may decide that theo whole projecct is infeasible. -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: wlEmoticon-smile[1].png Type: image/png Size: 1046 bytes Desc: not available URL: From adacore at marino.st Mon Jun 1 10:58:59 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:58:59 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> Message-ID: <556C1ED3.3000303@marino.st> On 6/1/2015 10:53, Jay K wrote: > make-dist makes everything. > It makes a "min" distribution and an "all" distribution. > Hopefully you can read it through it makes semi-reasonable sense. yes, I already figured this out. > This is all my doing -- which is to say, I quite like it, > but not necessarily anyone else. > min as I recall, is just the compiler: cm3, cm3cg, mklib, config, > m3core, libm3 > (mklib is only for NT386 target, but it is small and innocuous) > all is everything, including min > I don't like the confusion and purported complexity of breaking > things down into several medium sized packages, with dependencies. > I want more of a "one stop shop", granted, a big one, that > gives everyone stuff they won't use. > But I compromised and it makes to things. Right now, I'm assuming I can write an "install target" that copies from $STAGE/$DISTRIBUTION to where it needs to be installed in ports. If so, this approach is fine. Simpler even. > > cvs is pretty awful. > I'm still more productive in it than git, but I expect that will change. > But ok. cvsup can work. The error is for a trivial thing. I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and hopefully that will work. I don't know quake language. (I don't know why IsDarwin does't work because it's defined in the new cm3 distibution's cm3cfg.common.) Let's see how it goes. John From jay.krell at cornell.edu Mon Jun 1 10:53:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 08:53:00 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C16E7.505@marino.st> References: <556700B0.8060500@marino.st>, <5568421D.3050209@marino.st>, , <20150529155833.8454664b88391d93708e8f85@elego.de>, , <55687266.3090400@marino.st>, , <20150529162028.388911007a614006049beb42@elego.de>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> Message-ID: make-dist makes everything. It makes a "min" distribution and an "all" distribution. Hopefully you can read it through it makes semi-reasonable sense. This is all my doing -- which is to say, I quite like it, but not necessarily anyone else. min as I recall, is just the compiler: cm3, cm3cg, mklib, config, m3core, libm3 (mklib is only for NT386 target, but it is small and innocuous) all is everything, including min I don't like the confusion and purported complexity of breaking things down into several medium sized packages, with dependencies. I want more of a "one stop shop", granted, a big one, that gives everyone stuff they won't use. But I compromised and it makes to things. It makes roughly speaking .tar.gz files. Or .zips for some targets. Or compresed tar, depending on what compressor it can find, and bundles them into .deb files. But .deb files are little more than .tar.gz or .tar.bz2 or .tar.xz upgrade.sh also builds everything When I wrote it to upgrade.py I missed that significant aspect, and it only builds the compiler. Perhaps I should rename it to upgrade-min.py and then have upgrade-all.py. You can mimic upgrade.sh with ugprade.py && do-cm3-all.py realclean skipgcc && do-cm3-all.py buildship (skipgcc is just an optimization to avoid some duplicate work) I never use any of the .sh files. I found them to hard to understand and not portable enough. i.e. I'd rather install Python on Windows than Cygwin. I suspect Bourne shell is just not worth learning to program in. I had rewritten them in .cmd for Windows, but wanted one set of scripts. cvs is pretty awful. I'm still more productive in it than git, but I expect that will change. But ok. cvsup can work. The error is for a trivial thing. - Jay > Date: Mon, 1 Jun 2015 10:25:11 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 10:15, Jay K wrote: > > This is very good progress. > > Thank you for your speed and patience. > > > > > > And the error is kind of minor and uninteresting. Yes, I understand, it > > is fatal. Everything should just work to completion. > > > > > > See IsDarwin here: > > > > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > > > > cvsup is fiddling around how to use zlib. > > See here: > > https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile > > > > > > and surely cvsup isn't interesting any more? Who is using cvs? > > NetBSD and OpenBSD for starters. > > > > > You can delete m3-tools/cvsup. > > or change the scripts to skip it > > or update cm3cfg.common to be more up to date. > > I'd like to keep it, it's the best source for cvsup. > > I thought make-dist.py was just going to bootstrap (e.g. minimal > required). Is it actually building everything? > > > I was intending to run do-cm3-min.sh buildship, do-cm3-std.sh buildship, > do-cm3-caltech-parser.sh buildship after make-dist.py completed. > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 11:11:06 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:11:06 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1ED3.3000303@marino.st> References: <556700B0.8060500@marino.st>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> Message-ID: <556C21AA.70501@marino.st> On 6/1/2015 10:58, John Marino wrote: > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > hopefully that will work. I don't know quake language. quake doesn't know "False". If "0" doesn't work, I'll have to break down and come up with a patch to remove these last 3 lines. :) From jay.krell at cornell.edu Mon Jun 1 11:13:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 09:13:41 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C21AA.70501@marino.st> References: <556700B0.8060500@marino.st>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, ,,<556885FD.6010800@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, ,,<55689065.1040207@marino.st>, ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, ,, <556B213E.8030108@marino.st>,,, , ,, <556B4BE9.1000902@marino.st>,,, , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, ,,, ,,<556C1218.3000103@marino.st> , , <556C16E7.505@marino.st> , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st> Message-ID: try FALSE Here is some sample code: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common A patch shouldn't be needed but... hopefully soon I can try to duplicate what you are doing.. - Jay > Date: Mon, 1 Jun 2015 11:11:06 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 10:58, John Marino wrote: > > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > > hopefully that will work. I don't know quake language. > > quake doesn't know "False". If "0" doesn't work, I'll have to break > down and come up with a patch to remove these last 3 lines. :) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Mon Jun 1 11:24:56 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 1 Jun 2015 11:24:56 +0200 (CEST) Subject: [M3devel] duplicate subscription In-Reply-To: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> References: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> Message-ID: On Mon, 1 Jun 2015, Olaf Wagner wrote: > On Mon, 1 Jun 2015 09:11:22 +0200 (CEST) > Henning Thielemann wrote: > >> >> It seems that I am subscribed twice to this list. How can I find out the >> according e-mail addresses? > > Mailing list information can be found at > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel I know how to unsubscribe, but in order to do so I have to know the precise e-mail address I am subscribed with. I don't know them. :-( From adacore at marino.st Mon Jun 1 11:26:37 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:26:37 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , , <55689065.1040207@marino.st>, , , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , <556B213E.8030108@marino.st>, , , , , , <556B4BE9.1000902@marino.st>, , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , <556C1218.3000103@marino.st> , , <556C16E7.505@marino.st> , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st> Message-ID: <556C254D.7070306@marino.st> On 6/1/2015 11:13, Jay K wrote: > try FALSE > > Here is some sample code: > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > A patch shouldn't be needed but... hopefully soon I can try to duplicate > what you are doing.. Okay, I'll try "FALSE". That cm3cfg.common is pretty much what was installed at $STAGE/cm3-(all|min)-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/config/cm3cfg.common so I don't know why IsDarwin is not defined. === wait === okay, it failed again, same error (doesn't know IsDarwin), line 49 of m3-tools/cvsup/suplib/src/m3makefile So it seems we need to figure out why IsDarwin isn't known.... John From jay.krell at cornell.edu Mon Jun 1 11:29:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 09:29:54 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C254D.7070306@marino.st> References: <556700B0.8060500@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, , ,,<55689065.1040207@marino.st>, , ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , ,, <556B213E.8030108@marino.st>,,, , , , ,, <556B4BE9.1000902@marino.st>,,, , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , ,,, , , , <556C1218.3000103@marino.st> , ,,<556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> Message-ID: can you do like: export PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH cd /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib cm3 -commands -verbose cm3 -commands -verbose realclean cm3 -commands -verbose build - Jay > Date: Mon, 1 Jun 2015 11:26:37 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:13, Jay K wrote: > > try FALSE > > > > Here is some sample code: > > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > > A patch shouldn't be needed but... hopefully soon I can try to duplicate > > what you are doing.. > > Okay, I'll try "FALSE". > That cm3cfg.common is pretty much what was installed at > $STAGE/cm3-(all|min)-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/config/cm3cfg.common > so I don't know why IsDarwin is not defined. > > === wait === > > okay, it failed again, same error (doesn't know IsDarwin), line 49 of > m3-tools/cvsup/suplib/src/m3makefile > > So it seems we need to figure out why IsDarwin isn't known.... > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 11:44:02 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:44:02 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , <556B213E.8030108@marino.st>, , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , <556C1218.3000103@marino.st> , , , <556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> Message-ID: <556C2962.3020505@marino.st> On 6/1/2015 11:29, Jay K wrote: > can you do like: > export > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > cd > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > cm3 -commands -verbose > cm3 -commands -verbose realclean > cm3 -commands -verbose build It did not like "realclean" or "build": http://leaf.dragonflybsd.org/~marino/cvsupcmd.log John From jay.krell at cornell.edu Mon Jun 1 16:38:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 14:38:44 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C2962.3020505@marino.st> References: <556700B0.8060500@marino.st>, , ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , ,, <556B213E.8030108@marino.st>,,, , , , , , ,, <556B4BE9.1000902@marino.st>,,, , , , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , ,,, , , , , <556C1218.3000103@marino.st> , , ,,<556C16E7.505@marino.st> ,,, <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st> Message-ID: Sorry, multiple mistakes there by me. But I think I see the problem maybe. See it is always running mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 ? and never /mech/construction/mech/ptrees/default/lang/modula3/work/intermediate/compiler_with_{previous,self}/bin/{cm3,cm3cg} ? Do you have $CM3 set? Don't do that. It is supposed to be updating $PATH as it goes and finding cm3 and maybe cm3cg from there. But an environment variable CM3 will break it. https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py the function Setup That should print more, and maybe run and print the output of "where/which cm3" and "where/which cm3cg". and: https://github.com/modula3/cm3/blob/master/scripts/python/pylib.py: CM3 = ConvertPathForPython(getenv("CM3")) or "cm3" CM3 = SearchPath(CM3) I really just need to get and try your automation. You've made it available already? I'll have FreeBSD running locally "soon".. The right request by me would be to set PATH and CM3_INSTALL appropriately as make-dist.py would and run just building cvsup with -commands -verbose. Can you see how to do that? - Jay > Date: Mon, 1 Jun 2015 11:44:02 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:29, Jay K wrote: > > can you do like: > > export > > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > > cd > > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > > > cm3 -commands -verbose > > cm3 -commands -verbose realclean > > cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Jun 1 18:40:30 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 11:40:30 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C2962.3020505@marino.st> References: <556700B0.8060500@marino.st>, , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , <556B213E.8030108@marino.st>, , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , <556C1218.3000103@marino.st> , , , <556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> Message-ID: <556C8AFE.1050204@lcwb.coop> I am getting ready to write more on this, but a quick response: For the do-cm3-*.sh scripts, the options are realclean, build, buildship, etc. For cm3, the (supposedly) corresponding options are -realclean, -build, -buildship. This burns me every time I switch from one command form to the other. On 06/01/2015 04:44 AM, John Marino wrote: > On 6/1/2015 11:29, Jay K wrote: >> can you do like: >> export >> PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH >> cd >> /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib >> >> cm3 -commands -verbose >> cm3 -commands -verbose realclean >> cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 19:00:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 12:00:00 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1ED3.3000303@marino.st> References: <556700B0.8060500@marino.st>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> Message-ID: <556C8F90.9030907@lcwb.coop> On 06/01/2015 03:58 AM, John Marino wrote: > On 6/1/2015 10:53, Jay K wrote: >> make-dist makes everything. >> It makes a "min" distribution and an "all" distribution. >> Hopefully you can read it through it makes semi-reasonable sense. > > yes, I already figured this out. > >> This is all my doing -- which is to say, I quite like it, >> but not necessarily anyone else. >> min as I recall, is just the compiler: cm3, cm3cg, mklib, config, >> m3core, libm3 >> (mklib is only for NT386 target, but it is small and innocuous) >> all is everything, including min >> I don't like the confusion and purported complexity of breaking >> things down into several medium sized packages, with dependencies. >> I want more of a "one stop shop", granted, a big one, that >> gives everyone stuff they won't use. >> But I compromised and it makes to things. > > > Right now, I'm assuming I can write an "install target" that copies from > $STAGE/$DISTRIBUTION to where it needs to be installed in ports. If so, > this approach is fine. Simpler even. > > >> >> cvs is pretty awful. >> I'm still more productive in it than git, but I expect that will change. >> But ok. cvsup can work. The error is for a trivial thing. > > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > hopefully that will work. I don't know quake language. > Replace them with "", empty string. > (I don't know why IsDarwin does't work because it's defined in the new > cm3 distibution's cm3cfg.common.) > Yes, that surprises me too. suggests the new cm3cfg.common isn't getting interpreted. > Let's see how it goes. > John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 19:48:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 12:48:34 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , <556BD14D.5050507@lcwb.coop> Message-ID: <556C9AF2.7090704@lcwb.coop> On 06/01/2015 01:40 AM, Jay K wrote: > Yes, exactly, that's what I was trying to say, in my later/last mail. > > > Yes the compability is definitely two-way. > There is a one to one mapping back and forth. > cm3 goes with cm3cg; cm3cg goes with cm3. > Neither is compatible with older nor necessarily newer versions. > > > They would be statically linked together if not for the GPL. > > > When I first placed m3cc in pkginfo.txt years ago, I was only thinking > of build order and, like, "override". > > > m3cc imports nothing and nothing imports m3cc. > In the Modula-3 module/library/interface sense. > > > It can be built when you have nothing. > So I figured put it nearly first. > > > Only now today did I consider the shipping order. > It should be shipped adjacent to cm3, either right before or right after. > > > If you split build and ship, then it can still be built early, > or at almost any time. > Yes, this is the way I want to do building. First build everything you want to without putting any of it into use as build tools. Then put it all into use. One annoyance we currently have is that the do-cm3.*.sh scripts have, perhaps we should call them subcommands, like realclean, build, ship, buildship, etc. But the cm3 command treats their counterparts as command line options, -realclean, -build, -ship, -buildship, etc. I would like to consistify this. Removing the hyphen from cm3 would create ambiguities with file names, so it would mean adding the hyphen to the scripts. Much more serious is that similarly-named options do not have the same meaning in the scripts and in cm3. cm3 -build followed by cm3 -ship works as I expect, but do-cm3-*.sh build causes me nothing but trouble, and I learned long ago to avoid it. Instead, I always use do-cm3-*.sh buildship. This ships almost everything as it is built, but usually only cm3 and cm3cg executables matter, and today, they are specifically excepted from shipping, if you do not set INSTALL_CM3_IN_BIN="yes". I am writing from memory, but I think the problem with do-cm3-*.sh build is that build also uses overrides, which I barely understand, except that they, at best make things unshippable, and I think there was some other problem too. If we made the override thing a separate option to the scripts, so just build would neither ship nor override, I think that would satisfy my need, at least for the time being. And it would be consistent. Actually, I think we do want to use just-rebuilt packages when linking to them. Only executables and things they link in (possibly dynamically) need to be left alone until a later ship step. This usually only means cm3, cm3cg, and, if dynamically linked in, m3core. But there are others to think about, e.g., m3bundle, stubgen, netobjgen. > > But we have one ordering for build and ship, so let the shipping order decide. > > > I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. I have no objection to addressing the copy-over-an-executing-executable problem in a different way, and no objection to getting rid of the environment variable. But I do want it, for now at least, to be possible to get it to not ship either cm3 or cm3cg, even with the buildship option. If it just unconditionally doesn't ship them, that is fine. Before I added the variable check for the cm3cg package, it was unconditionally shipping it. > I still think it broke upgrade.py. > I can workaround. But I'd like to remove the environment variable check. > > > There are many order dependencies in build and ship. This is just one. > > > - Jay > > > > > > Date: Sun, 31 May 2015 22:28:13 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; adacore at marino.st > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > > to fail regardless of the order of building the two. > > > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > > it should work, because m3cc contains no Modula-3 code, and the new > > cm3 would not, when building it, run cm3cg. After that, there would > > be a consistent set. for further compiles. But any package containing > > M3 code built between cm3 and m3cc would fail. > > > > On 05/31/2015 05:15 PM, Jay wrote: > > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > > > - Jay > > > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > >> Should I put it back? From the last response I'm guessing that wasn't > > >> the problem. > > >> > > >> John > > >> > > >> > > >> On 5/31/2015 19:09, Jay K wrote: > > >>> Oh probably the problem is using the backend premature actually. > > >>> Darn, I can't help right now. > > >>> > > >>> - Jay > > >>> > > >>> > > >>> > > >>> ------------------------------------------------------------------------ > > >>> From: jay.krell at cornell.edu > > >>> To: adacore at marino.st; m3devel at elegosoft.com > > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > >>> (0x100), expected 0x110 > > >>> > > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > > >>> but my read was it didn't try that. > > >>> So.. with INSTALL_CM3_IN_BIN=yes? > > >>> I need to try this all, and "fix" it to set that. The whole thing in > > >>> m3cc/src/m3makefile is relatively > > >>> new vs. when I was actively using all this. :( > > >>> > > >>> - Jay > > >>> > > >>> > > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > > >>>> From: adacore at marino.st > > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > >>> (0x100), expected 0x110 > > >>>> > > >>>> On 5/31/2015 11:53, Jay K wrote: > > >>>>> John, have you tried make-dist.py? > > >>>>> i.e. > > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > > >>>>> > > >>>>> While it might not be exactly what you need, it should demonstrate the > > >>>>> elements. > > >>>>> > > >>>>> I will try to try this all soon, really, I hope so. > > >>>>> > > >>>>> It starts with a working compiler, does no in-place updates, build into > > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > > >>>>> It does "min" and "all". > > >>>>> > > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > > >>>>> > > >>>>> - Jay > > >>>> > > >>>> Hi Jay, > > >>>> I'm getting the same error as always using make-dist script: > > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > > >>>> > > >>>> Thanks, > > >>>> John > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 21:47:48 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 19:47:48 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C9AF2.7090704@lcwb.coop> References: <556700B0.8060500@marino.st>,<55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , , , <556BD14D.5050507@lcwb.coop>, , <556C9AF2.7090704@lcwb.coop> Message-ID: Sorry I'm rushing here..: Imho, build should build and ship should ship and buildship should do both. If that isn't the case, we should fix. Then, if you don't want to ship something, don't ship it. Not, ship it and set the environment variable. >implied: What is override? There are essentially two package pools, or sets of libraries, to chose from. They are roughly speaking the installed/shipped stuff: /usr/local/cm3/pkg/m3core/target (obviously it can vary) and the locally built but not yet shipped stuff, I call them: $HOME/dev2/cm3/m3-libs/m3core/target (obviously it can vary) I also call this "the object tree", i.e. the tree with a bunch of .obj/.o files. Or the "intermediates" tree, i.e. a bunch of outputs that are pruned out by "setup", and the outputs that are ultimately interesting The default is the first, override means to use the second. Then, the idea is that only stuff built against the install is safe to overwrite/add to the install. But I think is missing there is that, well, if I've built the entire system with overrides and I'm going to ship the entire system, then ship is safe. This isn't the best approach imho. Arguably better approaches are: always have a 2-level search first the object tree then the install tree This breaks people who switch around between incompatible changes in m3core and random application development, with the same source/object tree. However maybe that is just tough. The object tree doesn't have to be in the source tree. You should be able to say, like: symlinktree /usr/local/cm3 /working-on-feature1 export/set CM3_SOMETHING=/working-on-feature1 and case I change cm3 also export/set PATH=/working-on-feature1/bin:$PATH and then copy on write as you ship and then to switch to some other work: symlinktree /usr/local/cm3 /working-on-feature2 export/set CM3_SOMETHING=/working-on-feature2 and case I change cm3 also export/set PATH=/working-on-feature2/bin:$PATH and then copy on write as you ship, different stuff That is a working model that I advocate but don't yet follow. Though make-dist kind of does. make-dist copies the compiler to the new location and then does buildship. In this model, you don't have overrides and you always build/ship. And if you mess up, you can start over easily. But I haven't actually established workflow here so I'm just talking too much. : A real analog in other people's workflows is: mkdir /obj/working-on-feature1 cd /obj/working-on-feature1 /src/configure ... make mkdir /obj/working-on-feature2 cd /obj/working-on-feature2 /src/configure ...potentially different .. make .sh build should not ship. Agreed. Currently if you are going to ship, you can't override. make-dist.py uses a destdir-like approach. There is some contention perhaps of scenarios of "I'm just building a little, small safe changes" and "I'm rebuilding everything, and possibly messing up". I really want the environment variable checks to go away. - Jay > Date: Mon, 1 Jun 2015 12:48:34 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 06/01/2015 01:40 AM, Jay K wrote: > > Yes, exactly, that's what I was trying to say, in my later/last mail. > > > > > > Yes the compability is definitely two-way. > > There is a one to one mapping back and forth. > > cm3 goes with cm3cg; cm3cg goes with cm3. > > Neither is compatible with older nor necessarily newer versions. > > > > > > They would be statically linked together if not for the GPL. > > > > > > When I first placed m3cc in pkginfo.txt years ago, I was only thinking > > of build order and, like, "override". > > > > > > m3cc imports nothing and nothing imports m3cc. > > In the Modula-3 module/library/interface sense. > > > > > > It can be built when you have nothing. > > So I figured put it nearly first. > > > > > > Only now today did I consider the shipping order. > > It should be shipped adjacent to cm3, either right before or right after. > > > > > > If you split build and ship, then it can still be built early, > > or at almost any time. > > > > Yes, this is the way I want to do building. First build everything you want to > without putting any of it into use as build tools. Then put it all into use. > > One annoyance we currently have is that the do-cm3.*.sh scripts have, perhaps > we should call them subcommands, like realclean, build, ship, buildship, etc. > But the cm3 command treats their counterparts as command line options, > -realclean, -build, -ship, -buildship, etc. I would like to consistify > this. Removing the hyphen from cm3 would create ambiguities with file names, > so it would mean adding the hyphen to the scripts. > > Much more serious is that similarly-named options do not have the same meaning > in the scripts and in cm3. cm3 -build followed by cm3 -ship works as I expect, > but do-cm3-*.sh build causes me nothing but trouble, and I learned long ago to > avoid it. Instead, I always use do-cm3-*.sh buildship. This ships almost everything > as it is built, but usually only cm3 and cm3cg executables matter, and today, they are > specifically excepted from shipping, if you do not set INSTALL_CM3_IN_BIN="yes". > > I am writing from memory, but I think the problem with do-cm3-*.sh build is that > build also uses overrides, which I barely understand, except that they, at best > make things unshippable, and I think there was some other problem too. If we > made the override thing a separate option to the scripts, so just build would > neither ship nor override, I think that would satisfy my need, at least for the > time being. And it would be consistent. > > Actually, I think we do want to use just-rebuilt packages when linking to them. > Only executables and things they link in (possibly dynamically) need to be > left alone until a later ship step. This usually only means cm3, cm3cg, > and, if dynamically linked in, m3core. But there are others to think about, > e.g., m3bundle, stubgen, netobjgen. > > > > > But we have one ordering for build and ship, so let the shipping order decide. > > > > > > I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. > > I have no objection to addressing the copy-over-an-executing-executable problem > in a different way, and no objection to getting rid of the environment variable. > But I do want it, for now at least, to be possible to get it to not ship either > cm3 or cm3cg, even with the buildship option. If it just unconditionally doesn't > ship them, that is fine. Before I added the variable check for the cm3cg package, > it was unconditionally shipping it. > > > I still think it broke upgrade.py. > > I can workaround. But I'd like to remove the environment variable check. > > > > > > There are many order dependencies in build and ship. This is just one. > > > > > > - Jay > > > > > > > > > > > Date: Sun, 31 May 2015 22:28:13 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; adacore at marino.st > > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > > > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > > > to fail regardless of the order of building the two. > > > > > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > > > it should work, because m3cc contains no Modula-3 code, and the new > > > cm3 would not, when building it, run cm3cg. After that, there would > > > be a consistent set. for further compiles. But any package containing > > > M3 code built between cm3 and m3cc would fail. > > > > > > On 05/31/2015 05:15 PM, Jay wrote: > > > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > > > > > - Jay > > > > > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > > > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > > >> Should I put it back? From the last response I'm guessing that wasn't > > > >> the problem. > > > >> > > > >> John > > > >> > > > >> > > > >> On 5/31/2015 19:09, Jay K wrote: > > > >>> Oh probably the problem is using the backend premature actually. > > > >>> Darn, I can't help right now. > > > >>> > > > >>> - Jay > > > >>> > > > >>> > > > >>> > > > >>> ------------------------------------------------------------------------ > > > >>> From: jay.krell at cornell.edu > > > >>> To: adacore at marino.st; m3devel at elegosoft.com > > > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > > > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > > >>> (0x100), expected 0x110 > > > >>> > > > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > > > >>> but my read was it didn't try that. > > > >>> So.. with INSTALL_CM3_IN_BIN=yes? > > > >>> I need to try this all, and "fix" it to set that. The whole thing in > > > >>> m3cc/src/m3makefile is relatively > > > >>> new vs. when I was actively using all this. :( > > > >>> > > > >>> - Jay > > > >>> > > > >>> > > > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > > > >>>> From: adacore at marino.st > > > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > > > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > > >>> (0x100), expected 0x110 > > > >>>> > > > >>>> On 5/31/2015 11:53, Jay K wrote: > > > >>>>> John, have you tried make-dist.py? > > > >>>>> i.e. > > > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > > > >>>>> > > > >>>>> While it might not be exactly what you need, it should demonstrate the > > > >>>>> elements. > > > >>>>> > > > >>>>> I will try to try this all soon, really, I hope so. > > > >>>>> > > > >>>>> It starts with a working compiler, does no in-place updates, build into > > > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > > > >>>>> It does "min" and "all". > > > >>>>> > > > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > > > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > > > >>>>> > > > >>>>> - Jay > > > >>>> > > > >>>> Hi Jay, > > > >>>> I'm getting the same error as always using make-dist script: > > > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > > > >>>> > > > >>>> Thanks, > > > >>>> John > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Jun 2 00:16:57 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 1 Jun 2015 18:16:57 -0400 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> Message-ID: <20150601221657.GA4281@topoi.pooq.com> On Mon, Jun 01, 2015 at 07:47:48PM +0000, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. Make sense to me. -- hendrik From hendrik at topoi.pooq.com Tue Jun 2 00:23:19 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 1 Jun 2015 18:23:19 -0400 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> Message-ID: <20150601222319.GA4410@topoi.pooq.com> On Mon, Jun 01, 2015 at 07:47:48PM +0000, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. This would mean there would have to be some kind of loading dok for things that have been built but not yet shipped. It probably exists already. -- hendrik From rodney_bates at lcwb.coop Tue Jun 2 02:54:45 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 19:54:45 -0500 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, , <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , , , <556BD14D.5050507@lcwb.coop>, , <556C9AF2.7090704@lcwb.coop> Message-ID: <556CFED5.80002@lcwb.coop> Here is a short-term proposal (i.e., without major reorganization) for the do-cm3*.sh scripts: 1) 'build' only builds, as we seem to agree it should. 2) a new option 'override' (and only 'override') causes an override build 3) a new option 'partialship' ships, as each package is done, things that will be needed to compile another package that does a quake import on the just-built package (I think this means static library, if any, and .M3WEB), but does not ship things that will be used to execute the just-built package (I think this means executable or dynamic library). I'm not sure right off hand which ship group things like interface source files, etc. belong in. Plus, just as conveniences: 4) buildship means the same as build and ship (I think this is already the case.) 5) buildpartialship means the same as build and partialship 6) All these options at least allow a leading hyphen, so they are like the cm3 command. I guess cm3 should also have a -shippartial On 06/01/2015 02:47 PM, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 2 09:11:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 2 Jun 2015 07:11:44 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, ,,,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , ,,,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , , , , <556B213E.8030108@marino.st>, , , , ,,, , , , , , , <556B4BE9.1000902@marino.st>, , , , ,,, , ,,,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , ,,,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , ,,,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , ,,,,, , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: John, this and your other problems should all be fixed now. I was able to reproduce them mostly.I didn't actually reproduce the cvsup problem, but I see in the log you did set CM3, and I did test with that, and hit the cm3cg version mismatch, and the problem is pretty clear -- my mistake. See: https://github.com/modula3/cm3/commit/a149d4b5e715d89b076c92e0b41d74f9b5ff56e4 https://github.com/modula3/cm3/commit/769784562d0e45b968e515c1412ed9247ec050ef https://github.com/modula3/cm3/commit/a0e44b08e687a8bba677c299ffa459993525944c Thank you for your patience and diligence. I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && make-dist.py or skip right to make-dist.py.There was an additional problem skipping right to make-dist.py that I fixed, at least for Darwin. I don't use any of the .sh files -- I don't want to read/maintain/learn that programming language as I suspect it isn't particularly good. - Jay From: jay.krell at cornell.edu To: adacore at marino.st CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 Date: Mon, 1 Jun 2015 14:38:44 +0000 Sorry, multiple mistakes there by me. But I think I see the problem maybe. See it is always running mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 ? and never /mech/construction/mech/ptrees/default/lang/modula3/work/intermediate/compiler_with_{previous,self}/bin/{cm3,cm3cg} ? Do you have $CM3 set? Don't do that. It is supposed to be updating $PATH as it goes and finding cm3 and maybe cm3cg from there. But an environment variable CM3 will break it. https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py the function Setup That should print more, and maybe run and print the output of "where/which cm3" and "where/which cm3cg". and: https://github.com/modula3/cm3/blob/master/scripts/python/pylib.py: CM3 = ConvertPathForPython(getenv("CM3")) or "cm3" CM3 = SearchPath(CM3) I really just need to get and try your automation. You've made it available already? I'll have FreeBSD running locally "soon".. The right request by me would be to set PATH and CM3_INSTALL appropriately as make-dist.py would and run just building cvsup with -commands -verbose. Can you see how to do that? - Jay > Date: Mon, 1 Jun 2015 11:44:02 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:29, Jay K wrote: > > can you do like: > > export > > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > > cd > > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > > > cm3 -commands -verbose > > cm3 -commands -verbose realclean > > cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Tue Jun 2 09:19:33 2015 From: adacore at marino.st (John Marino) Date: Tue, 02 Jun 2015 09:19:33 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>, , , , , , , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: <556D5905.4010304@marino.st> On 6/2/2015 09:11, Jay K wrote: > John, this and your other problems should all be fixed now. > > I was able to reproduce them mostly. > I didn't actually reproduce the cvsup problem, but I see in the log you > did set CM3, and I did test with that, and hit the cm3cg version > mismatch, and the problem is pretty clear -- my mistake. Thanks Jay! Don't worry about the mistake. I'm just happy that my use case was able to flush out some issues and get some interesting discussion going. Sure it's been hard-going from my side but the final result is an improvement for everyone. I'll see what I can test at night in a VM but I don't get home back to my "real" machine until late Friday night (CEDT). But after looking over the commits this seems promising. John From wagner at elegosoft.com Tue Jun 2 11:36:11 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 2 Jun 2015 11:36:11 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556CFED5.80002@lcwb.coop> References: <556700B0.8060500@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> Message-ID: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> On Mon, 01 Jun 2015 19:54:45 -0500 "Rodney M. Bates" wrote: > Here is a short-term proposal (i.e., without major reorganization) > for the do-cm3*.sh scripts: > > 1) 'build' only builds, as we seem to agree it should. > 2) a new option 'override' (and only 'override') causes an override build > 3) a new option 'partialship' ships, as each package is done, things that > will be needed to compile another package that does a quake import on the > just-built package (I think this means static library, if any, and .M3WEB), > but does not ship things that will be used to execute the just-built package > (I think this means executable or dynamic library). I'm not sure right off > hand which ship group things like interface source files, etc. belong in. I don't know how you will implement this and how it should fit into the M3 package concept, unless you ship to a completely different location, i.e. another package pool (but cm3 currently just supports one). In my opinion the package system relies on the invariant that a shipped (installed) package is completely consistent, i.e. the source code interfaces correspond to the intermediate code interfaces and to the binary equivalent of them (static and dynamic libraries). If I understand you correctly, you want to give up this consistency in favour of being able to do an easier bootstrap of the CM3 system. But bootstrapping a complex system is never easy and usually requires some special or tricky steps. Ordinary use of the system, i.e. all other applications, that just build on the standard tools, don't need this kind of steps. If I could design (and implement) a better cm3 builder, I'd have one with multiple package pools (shipping destinations, locations of installed packages) and an integrated builder that knows all about the package dependencies (so that we haven't to do that in scripts). The two points are the only shortcomings in the cm3 build system in my opinion. The prjm tools of the ComPact sources was a step in this direction: it can handle multiple pools of packages and their dependencies, but is language independent, therefore too complex, and never got integrated into the compiler builder. As for the shell scripts, I'm surprised that they have survived so long; they just got created on the fly to support the building and publication of the CM3 system as open source when we got the sources from Critical Mass. They were never intended as a general user interface, but only as tools for the CM3 maintainers, and later got used in the Hudson CI. BTW, if I find some time and access to suitable machines again, I'd really like to set up a new CI system on Jenkins interfacing with the Github infrastruture. But I won't make any promises here. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From lists at darko.org Tue Jun 2 14:05:48 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 2 Jun 2015 05:05:48 -0700 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> References: <556700B0.8060500@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> Message-ID: Why can't we design and build a cm3 builder that supports these requirements? Relying on scripts seems entirely broken and an anathema to the portability of the CM3 system. There seems to be enough experience to specify and design such a system and building it doesn't sound like it would be very difficult. I'm sure there would be enough volunteers to implement it since people's productivity is obviously being affected. Wouldn't we want to fix the problem for once and for all instead of endlessly hacking scripts? Having a robust, reliable and portable end-to-end build system would also make it easier for people wanting to port and enhance the compiler, which would be a boost for the project. On Tue, Jun 2, 2015 at 2:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > > > Here is a short-term proposal (i.e., without major reorganization) > > for the do-cm3*.sh scripts: > > > > 1) 'build' only builds, as we seem to agree it should. > > 2) a new option 'override' (and only 'override') causes an override build > > 3) a new option 'partialship' ships, as each package is done, things that > > will be needed to compile another package that does a quake import > on the > > just-built package (I think this means static library, if any, and > .M3WEB), > > but does not ship things that will be used to execute the just-built > package > > (I think this means executable or dynamic library). I'm not sure > right off > > hand which ship group things like interface source files, etc. > belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other > applications, > that just build on the standard tools, don't need this kind of steps. > > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 2 18:18:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 02 Jun 2015 11:18:03 -0500 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> Message-ID: <556DD73B.9050601@lcwb.coop> On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org From dmuysers at hotmail.com Tue Jun 2 18:39:35 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 2 Jun 2015 18:39:35 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556DD73B.9050601@lcwb.coop> References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop><20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: golang is an example of good package management. It`s interaction of the builder with github repositories is an interesting idea. -----Original Message----- From: Rodney M. Bates Sent: Tuesday, June 02, 2015 6:18 PM To: Olaf Wagner Cc: m3devel Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 2 19:48:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 2 Jun 2015 17:48:13 +0000 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop><20150602113611.66de95acf8cdb242c6672605@elegosoft.com>, <556DD73B.9050601@lcwb.coop>, Message-ID: > Yes, absolutely. We need a greatly reworked build system for multiple packages. > For building within a package, cm3's existing build is really quite sophisticated. Exactly my thoughts for years. quake is nice and declarative for each package, and currently does nothing for multiple packages. How far would we get with the following: include_dir to stitch together packages, the same as it stitches directories within packages. "Flush" the state at each Library() or Program() invocation? Library and Program have to be last currently? So that would work? Or they can be anywhere? Furthermore, target directory either be: 1) ./target -- might be what it'd do now, but is wrong choice 2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. cd anywhere, that is still it 3) under some new new root, like go up from cwd until you find special marker file; this would be building outside source tree entirely 4) same as currently, wherever is Library()/Program(), go ../ If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"? Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir. it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib. It looks like:dirs file: DIRS=a \ b \ c sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake I use this system a lot and quite like it.Though it is obscure. The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad. I think scons works this way too, the language being Python. Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program. Make sense? Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.) - Jay From: dmuysers at hotmail.com To: rodney.m.bates at acm.org; wagner at elegosoft.com Date: Tue, 2 Jun 2015 18:39:35 +0200 CC: m3devel at elegosoft.com Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 golang is an example of good package management. It`s interaction of the builder with github repositories is an interesting idea. -----Original Message----- From: Rodney M. Bates Sent: Tuesday, June 02, 2015 6:18 PM To: Olaf Wagner Cc: m3devel Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Jun 2 20:59:51 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 2 Jun 2015 14:59:51 -0400 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: <20150602185951.GA6137@topoi.pooq.com> On Tue, Jun 02, 2015 at 05:48:13PM +0000, Jay K wrote: > > Yes, absolutely. We need a greatly reworked build system for multiple packages. > > For building within a package, cm3's existing build is really quite sophisticated. > > Exactly my thoughts for years. quake is nice and declarative for each package, and currently > does nothing for multiple packages. > > > How far would we get with the following: > > > include_dir to stitch together packages, the same as it stitches > directories within packages. > > "Flush" the state at each Library() or Program() invocation? > > Library and Program have to be last currently? So that would work? > Or they can be anywhere? > > > Furthermore, target directory either be: > 1) ./target -- might be what it'd do now, but is wrong choice Probably the wrong place. ./cm3target might be slightly better. Or make it not hidden. Or have a quake conmend to set it. > 2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. > cd anywhere, that is still it > 3) under some new new root, like go up from cwd until you find > special marker file; this would be building outside source tree entirely Outside the source tree entirely would be good. I'd really like to be able to compile a source tree that's on a read-only file system. Or on a shadow file system. > 4) same as currently, wherever is Library()/Program(), go ../ > > If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"? > > Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir. > > it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib. > > It looks like:dirs file: DIRS=a \ b \ c > sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake > I use this system a lot and quite like it.Though it is obscure. > The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad. > > I think scons works this way too, the language being Python. Writing the cm3 stanzas to tell scons how to use cm3 sould be useful, too. > > Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program. > > Make sense? > Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.) > > - Jay > > > > > From: dmuysers at hotmail.com > To: rodney.m.bates at acm.org; wagner at elegosoft.com > Date: Tue, 2 Jun 2015 18:39:35 +0200 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > > > golang is an example of good > package management. > It`s interaction of the builder with github repositories is an interesting > idea. > > -----Original Message----- > From: Rodney M. Bates > Sent: Tuesday, June 02, 2015 6:18 PM > To: Olaf Wagner > Cc: m3devel > Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG > version stamp (0x100), expected 0x110 > > > > On 06/02/2015 04:36 AM, Olaf Wagner wrote: > > On Mon, 01 Jun 2015 19:54:45 -0500 > > "Rodney M. Bates" wrote: > > > >> Here is a short-term proposal (i.e., without major > reorganization) > >> for the do-cm3*.sh scripts: > >> > >> 1) 'build' only builds, as we seem to agree it should. > >> 2) a new option 'override' (and only 'override') causes an > override build > >> 3) a new option 'partialship' ships, as each package is done, > things that > >> will be needed to compile another > package that does a quake import on the > >> just-built package (I think this > means static library, if any, and .M3WEB), > >> but does not ship things that will > be used to execute the just-built package > >> (I think this means executable or > dynamic library). I'm not sure right off > >> hand which ship group things like > interface source files, etc. belong in. > > > > I don't know how you will implement this and how it should fit into > the > > M3 package concept, unless you ship to a completely different > location, > > i.e. another package pool (but cm3 currently just supports one). > > > > In my opinion the package system relies on the invariant that a > shipped > > (installed) package is completely consistent, i.e. the source > code > > interfaces correspond to the intermediate code interfaces and to > the > > binary equivalent of them (static and dynamic libraries). If I > understand > > you correctly, you want to give up this consistency in favour of > being > > able to do an easier bootstrap of the CM3 system. > > Yes, but we already must have a temporarily inconsistent state of > installation > today. This is a somewhat different kind of inconsistency, but > inconsistent, > nonetheless. > > I guess -partialship could do an additional final pass over the package > list, > doing a full ship. The inconsistent state would only last during a > single > user-requested step, which is about as short as it can get. My intent > was > that one would immediately do a full ship after a partialship, although > as > a separate command, thus restoring consistency. > > > > > > But bootstrapping a complex system is never easy and usually requires > some > > special or tricky steps. Ordinary use of the system, i.e. all other > applications, > > that just build on the standard tools, don't need this kind of > steps. > > > > And my proposal is a simple, short-term way to provide somewhat more > flexible > options for just such special or tricky steps. It will work the same > way as now, > if you don't use the new options, and would allow Jay to get rid of the > hated > INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special > steps. > > > If I could design (and implement) a better cm3 builder, I'd have one > with > > multiple package pools (shipping destinations, locations of > installed > > packages) and an integrated builder that knows all about the > package > > dependencies (so that we haven't to do that in scripts). The two > points > > are the only shortcomings in the cm3 build system in my opinion. > > The prjm tools of the ComPact sources was a step in this > direction: > > it can handle multiple pools of packages and their dependencies, > but > > is language independent, therefore too complex, and never got > integrated > > into the compiler builder. > > > > Yes, absolutely. We need a greatly reworked build system for multiple > packages. > For building within a package, cm3's existing build is really quite > sophisticated. > I don't know of any other that detects the need for recompilation > on declaration-granularity. But it does very little with > inter-package > problems -- just detection of link-time inconsistencies, with > hopelessly > uninformative error messages. (I once set out to improve them, but > it > required more info in the .mx and .m3x files, with more incompatible > changes, and I only got one message improved a little bit, before I > gave > up for the time.) > > I agree, reworking it all is very desirable. It will take a lot of > rework. > I think we all agree that some kind of multi-layer, multi-destination > scheme is needed. This will probably entail disentangling the > source > and build directories from being interspersed in each package, or > something > equally perturbing to the existing system. It isn't going to be a > little > patch. I think my proposal will be small to implement and have no > impact > on anything existing, if you don't use the new options. > > > > As for the shell scripts, I'm surprised that they have survived so > long; > > they just got created on the fly to support the building and > publication > > of the CM3 system as open source when we got the sources from > Critical > > Mass. They were never intended as a general user interface, but > only > > as tools for the CM3 maintainers, and later got used in the Hudson > CI. > > > > BTW, if I find some time and access to suitable machines again, > I'd > > really like to set up a new CI system on Jenkins interfacing with > the > > Github infrastruture. But I won't make any promises here. > > > > Yes, this would be very good. Priorities, priorities! :-(. > > > Olaf > > > > -- > Rodney Bates > rodney.m.bates at acm.org From wagner at elegosoft.com Tue Jun 2 21:19:52 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 2 Jun 2015 21:19:52 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556DD73B.9050601@lcwb.coop> References: <556700B0.8060500@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: <20150602211952.17478aefac4b66e13de704a9@elegosoft.com> On Tue, 02 Jun 2015 11:18:03 -0500 "Rodney M. Bates" wrote: > > > On 06/02/2015 04:36 AM, Olaf Wagner wrote: > > On Mon, 01 Jun 2015 19:54:45 -0500 > > "Rodney M. Bates" wrote: > > > >> Here is a short-term proposal (i.e., without major reorganization) > >> for the do-cm3*.sh scripts: > >> > >> 1) 'build' only builds, as we seem to agree it should. > >> 2) a new option 'override' (and only 'override') causes an override build > >> 3) a new option 'partialship' ships, as each package is done, things that > >> will be needed to compile another package that does a quake import on the > >> just-built package (I think this means static library, if any, and .M3WEB), > >> but does not ship things that will be used to execute the just-built package > >> (I think this means executable or dynamic library). I'm not sure right off > >> hand which ship group things like interface source files, etc. belong in. > > > > I don't know how you will implement this and how it should fit into the > > M3 package concept, unless you ship to a completely different location, > > i.e. another package pool (but cm3 currently just supports one). > > > > In my opinion the package system relies on the invariant that a shipped > > (installed) package is completely consistent, i.e. the source code > > interfaces correspond to the intermediate code interfaces and to the > > binary equivalent of them (static and dynamic libraries). If I understand > > you correctly, you want to give up this consistency in favour of being > > able to do an easier bootstrap of the CM3 system. > > Yes, but we already must have a temporarily inconsistent state of installation > today. This is a somewhat different kind of inconsistency, but inconsistent, > nonetheless. > > I guess -partialship could do an additional final pass over the package list, > doing a full ship. The inconsistent state would only last during a single > user-requested step, which is about as short as it can get. My intent was > that one would immediately do a full ship after a partialship, although as > a separate command, thus restoring consistency. OK, I think I have a better idea of what you want to do now. > > But bootstrapping a complex system is never easy and usually requires some > > special or tricky steps. Ordinary use of the system, i.e. all other applications, > > that just build on the standard tools, don't need this kind of steps. > > > > And my proposal is a simple, short-term way to provide somewhat more flexible > options for just such special or tricky steps. It will work the same way as now, > if you don't use the new options, and would allow Jay to get rid of the hated > INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > > > If I could design (and implement) a better cm3 builder, I'd have one with > > multiple package pools (shipping destinations, locations of installed > > packages) and an integrated builder that knows all about the package > > dependencies (so that we haven't to do that in scripts). The two points > > are the only shortcomings in the cm3 build system in my opinion. > > The prjm tools of the ComPact sources was a step in this direction: > > it can handle multiple pools of packages and their dependencies, but > > is language independent, therefore too complex, and never got integrated > > into the compiler builder. > > > > Yes, absolutely. We need a greatly reworked build system for multiple packages. > For building within a package, cm3's existing build is really quite sophisticated. > I don't know of any other that detects the need for recompilation > on declaration-granularity. But it does very little with inter-package > problems -- just detection of link-time inconsistencies, with hopelessly > uninformative error messages. (I once set out to improve them, but it > required more info in the .mx and .m3x files, with more incompatible > changes, and I only got one message improved a little bit, before I gave > up for the time.) > > I agree, reworking it all is very desirable. It will take a lot of rework. > I think we all agree that some kind of multi-layer, multi-destination > scheme is needed. This will probably entail disentangling the source > and build directories from being interspersed in each package, or something > equally perturbing to the existing system. It isn't going to be a little > patch. I think my proposal will be small to implement and have no impact > on anything existing, if you don't use the new options. I'm not against your proposal as a first or intermediate step. We have to remain pragmatic and improve things incrementally. If people really use the old shell scripts, I'm not against adapting them. Jay may do all the same things in Python even faster, but I always found that was an additional unnecessary dependency. On the other hand, Python can probably be installed in a couple of minutes on any system these days. I also agree that we would all like a multi-level fully integrated build system for sets of M3 packages, but that will be a lot of work and I won't be able to contribute much to it. Those who do the work will decide about the design and its capabilities. > > As for the shell scripts, I'm surprised that they have survived so long; > > they just got created on the fly to support the building and publication > > of the CM3 system as open source when we got the sources from Critical > > Mass. They were never intended as a general user interface, but only > > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > > > BTW, if I find some time and access to suitable machines again, I'd > > really like to set up a new CI system on Jenkins interfacing with the > > Github infrastruture. But I won't make any promises here. > > Yes, this would be very good. Priorities, priorities! :-(. It's not as improbable as me writing a new integrated build system though, as I've set up several complex CI systems for other projects. We'll se if I find the fime... Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Jun 3 03:58:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 01:58:10 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem Message-ID: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 04:51:45 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 02:51:45 +0000 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) Message-ID: Ideally: 1 32bit host could target 64bit target 2 32bit texts could be as large as the address space/OS allows. 3 ditto 64bit texts -- larger than 4GB 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. Currently only #4 is true. TEXTs for any target can only be about 500MB. To fix 2 and 3 requires m3front to use TInt more. This fixes #1 and likely breaks #4. Cutting the limit just a little. Can we special case somehow? Is there some construct we can use that has no stated limit, like 0..cnt - 1? Should/can we introduce one? jbook2:src jay$ git diff "../src/text/TextLiteral.i3"diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..4d16a44 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; CONST (* DIV BITSIZE should not be here! *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ otherwise we have: new source -> compiling TextLiteral.i3"../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large1 error encountered Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 08:17:49 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 08:17:49 +0200 Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>, , , , , , , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: <556E9C0D.5000403@marino.st> On 6/2/2015 09:11, Jay K wrote: > John, this and your other problems should all be fixed now. > > I was able to reproduce them mostly. > I didn't actually reproduce the cvsup problem, but I see in the log you > did set CM3, and I did test with that, and hit the cm3cg version > mismatch, and the problem is pretty clear -- my mistake. > > I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && > make-dist.py or skip right to make-dist.py. > There was an additional problem skipping right to make-dist.py that I > fixed, at least for Darwin. Hi Jay, I set up the VM last night and it completed the make-dist.py command. I have some questions / comments. comment 1: Using bin for config isn't a standard unix location. e.g. rather than /bin/config/AMD64_FREEBSD, it should be located at /etc/modula3/config/AMD64_FREEBSD (can it be configurable?) comment 2: Many of the binaries are might actually be example programs, e.g. cube, calculator, fisheye. Rather than /bin where they risk clashing with other programs, it might be advisable to stick them a /share/examples/modula3 (configurable?). comment 3: /www is not standard either. We would be this in /share/doc/modula3. Can it be configurable? Comments 1--3 might be related because www might have relative paths that need changing comment 4: I'd like a configurable option not to tar/gzip the products. In my case, I'm going to stage them manually so this is an unnecessary step. comment 5: the "min" distribution seems suitable to be packaged as a bootstrap question 6: In most use cases, other than intentionally recreating the bootstrap, I'm not going to need the "min" distribution. Is there any impact to building "all" only? In other words, does something else use "min" distribution? Thanks for the update! John From jay.krell at cornell.edu Wed Jun 3 09:18:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 07:18:41 +0000 Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) In-Reply-To: <556E9C0D.5000403@marino.st> References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>,,, , , , , , , , , ,,, , , <556B4BE9.1000902@marino.st>, , , , , ,,, , , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , ,,<556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, ,,, , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st> Message-ID: "Yes". 1) "config"'s user configurability is over-implied imho. It shouldn't all be considered config. It needs refactoring into bin and config. Possibly rewriting the stuff that has stood the test of a few year's time into native Modula-3. It used to be worse. Each file used to be standalone. I refactored -- all the "common" files. I think it is partly called config because if you look at the gcc source tree, it also has a config directory. https://github.com/modula3/cm3/tree/master/m3-sys/m3cc/gcc/gcc/config It should be called "target" perhaps. Heck, we could have bin//cm3cg, bin//cm3.cfg or bin/config.quake or such. Consider all the Perl and Python and sh you may or may not have in /usr/bin. Is it configurable? User editable? Maybe/kinda. Consider that /etc is often sh snippets. The line between code and data is somewhat blurry, and I am somewhat lazy. We could split the files into "less editable" and "more editable". The bulk of the text in the files is hard to understand and hard to edit. And then there is the question of updating. And when I make edits to "the defaults", which do I assume are carried with cm3, vs. which do I assume an update will leave alone? Basically, I'm lazy, and haven't perfectly evaluated or factored everything. It used to be worse. However, there is/was cminstall, which the user is/was meant to run which would edit small parts. So maybe those very parts should be broken out into separate files? There another problem though. Which parts of "config"/cminstall need to be rerun when something else changes? You know, imagine I install cm3 and then I install X Windows. Installing X Windows implies a need to reconfigure. The full needs of a configuration system are related to a packaging system. Debian I believe handles this all well, after years of development focusing on these sorts of things. And then we'd have to handle all the other packaging systems. And nobody has been interested/motivated enough so far to really integrate so well into all the various packaging systems. But also, we target not just Unix. Yet we have almost the same directory structure on Unix and Windows. The difference is that on Windows we put the .dlls in bin. Should we use $HOME/etc or $HOME/config? And have /usr/local/cm3/bin probe around? I think I did put in some probing. To what extend do users just have private installs $HOME/cm3/bin vs. sharing binaries and having just split off config? What if they want to change the compiled code? To what extent are systems multi-user? So "yes" there is work to do here and we need help :) 2) Yes. Remember how I said I was lazy and made just one, or rather two, package sets? That is what you are seeing here. Olaf did put more thought/work into this and the .sh files do build a few different distribution sets. 3) I think so. Look for "www" in config? 1-3) yes. In particular, are you aware of the rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? I went with origin. I didn't have the energy or perhaps the buy-in to implement relink-upon-install approach. Previously we relied on either paths being correct *before* make install, or setting LD_LIBRARY_PATH. Both are bad solutions. There are I believe approx. 4 choices, for redistributable binaries that might go to a variety of setups: a) paths correct on original build machine b) LD_LIBARY_PATH c) origin d) relink upon install (leaving binaries not further relocatable by mv/cp). I went with c. So if you move samples around, you have to deal with that. bin/samples would seem easy enough though. Or heck, a peer of bin? lib? samplebin? samples? I think if we adopted automake/libtool, we'd get #c and it would be kinda good. Do cmake users use libtool? You know meta meta meta point -- we have our own little close to state of the art build system, but not quite state of the art. It works prety well and pretty portably. But not always as well as people would like. Between autoconf and Perl's method of port to every system, we are more like Perl, but also trying to find least common denominators that don't actually have to be ported. Does this make some sense? 4) Just command line parsing in make-dist.py. 5) yes 6) I don't understand. min is meant to be: 6a) a minimal toolset You can write hello world. cm3, cm3cg, mklib, config, m3core, libm3 text, threads not a gui This is subjective. The "resource" stuff should probably be here. 6b) It is meant to be enough to bootstrap another cm3 version from, older or newer. cm3 manages its dependencies carefully. It either carries things itself, or they are in min. For example, it doesn't use X Windows (or .i3 files describing it). upgrade.py/upgrade.sh assume min Now, granted, we can bootstrap from nothing as well. 7) You didn't ask. But I kinda think we should only have environment variables that start CM3 -- CM3 and start CM3_ actually. 8) Another large meta point -- this is all clearly imperfect, usually for lack of time or energy, not because we felt it was better this way. Though there is the angle I gave -- portability. Also, autoconf was probably less well established when most of this was written. The build/conf world seems still to be somewhat unsettled. 9) I have a larger unimplemented vision -- the system should be redistributed as one portable hard to read C or C++ .tar.gz. This doesn't work today, because "the frontend does layout" -- we essentially have 6 flavors of C: 1 posix 32bit little endian (x86, etc.) 2 posix 32bit big endian (ppc_darwin, sparc32) 3 posix 64bit little endian (amd64. etc.) 4 posix 64bit big endian (ppc64_darwin, sparc64) 5 Windows 32bit little endian (x86 mips PowerPC alpha, arm32, etc. ) 6 Windows 32bit big endian (hypothetical xbox 360? CE?) 7 Windows 64bit little endian (amd64, ia64, alpha64 etc.) 8 Windows 64bit big endian (hypothetical xbox 360?) We could today make 6 distributions of just C and bootstrap from one of them, picking the right one. Most 64bit systems can run 32bit, so strike those as bootstrap? But not quite, e.g. OpenBSD. I'd like the option for the frontend to defer layout to the backend, and for the C backend to output string expressions involving sizeof(size_t) or sizeof(char*). And, for win32 vs. posix, I'd like quake to compile both, and to generate a Makefile that picks the right one, or a .sh/.cmd. Oh, and jmpbuf size is in the generated C. I know how to fix that. bootstrap specifically could inflate it to the largest known case, which is quite large. Or, once we generate C++ for excpetion handling, jmpbuf should go away. - Jay > Date: Wed, 3 Jun 2015 08:17:49 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) > > On 6/2/2015 09:11, Jay K wrote: > > John, this and your other problems should all be fixed now. > > > > I was able to reproduce them mostly. > > I didn't actually reproduce the cvsup problem, but I see in the log you > > did set CM3, and I did test with that, and hit the cm3cg version > > mismatch, and the problem is pretty clear -- my mistake. > > > > I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && > > make-dist.py or skip right to make-dist.py. > > There was an additional problem skipping right to make-dist.py that I > > fixed, at least for Darwin. > > Hi Jay, > I set up the VM last night and it completed the make-dist.py command. I > have some questions / comments. > > comment 1: > Using bin for config isn't a standard unix location. > e.g. rather than /bin/config/AMD64_FREEBSD, it should be located > at /etc/modula3/config/AMD64_FREEBSD (can it be configurable?) > > comment 2: > Many of the binaries are might actually be example programs, e.g. cube, > calculator, fisheye. Rather than /bin where they risk clashing > with other programs, it might be advisable to stick them a > /share/examples/modula3 (configurable?). > > comment 3: /www is not standard either. We would be this in > /share/doc/modula3. Can it be configurable? > > Comments 1--3 might be related because www might have relative paths > that need changing > > comment 4: I'd like a configurable option not to tar/gzip the products. > In my case, I'm going to stage them manually so this is an unnecessary > step. > > comment 5: the "min" distribution seems suitable to be packaged as a > bootstrap > > question 6: In most use cases, other than intentionally recreating the > bootstrap, I'm not going to need the "min" distribution. Is there any > impact to building "all" only? In other words, does something else use > "min" distribution? > > Thanks for the update! > John > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 10:36:51 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 10:36:51 +0200 Subject: [M3devel] It works! In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st> Message-ID: <556EBCA3.3090601@marino.st> On 6/3/2015 09:18, Jay K wrote: > 1) "config"'s user configurability is over-implied imho. > [snip] > But also, we target not just Unix. > Yet we have almost the same directory structure on Unix and Windows. > The difference is that on Windows we put the .dlls in bin. It is possible this was misunderstood. It's important to release that cm3 will be installed nominally at /usr/local/bin. On a normal unix system, there could be over 500 packages all installing there. Things like configuration files (.conf) are nominally installed at /usr/local/etc/. The config directory is analogous to these. I will do the following in any case: A) move config from bin to etc// B) edit cm3.cfg to point to new location All I was suggesting is that the script do this for me. It can still install at bin/ by default, but it would be nice if it could install elsewhere too. > Should we use $HOME/etc or $HOME/config? Never. This does not work for packaging. E.g. prebuild binary packages that are installed (no building required) > 2) "example" progs > Yes. Remember how I said I was lazy and made just one, or rather two, > package sets? That is what you are seeing here. > Olaf did put more thought/work into this and the .sh files do build a > few different distribution sets. I am fine with what is built. Remember above that cm3 is installed with 500 other packages. Any program that is not essential and essentially there as an example could (and my opinion) should be installed in a place other than bin/. I don't care if it's installed outside of PATH because it probably won't be called much and absolute path is fine in this case. > 3) I think so. Look for "www" in config? > 1-3) yes. > In particular, are you aware of the > rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? I just remember that the current cm3 in ports, which is split up like I suggest, had some broken links because I moved these things around. I can't remember what is actually broken; I'd have to check again. The point was you can't just move stuff around without changing links, so the install script would have to do this if it supported alternative locations. > 4) > Just command line parsing in make-dist.py. Right, but currently I have to patch make-dist.py to make it not tar/gzip. If the script supported the option, I could avoid that patch. > 5) yes > 6) I don't understand. > min is meant to be: > 6a) a minimal toolset > You can write hello world. > cm3, cm3cg, mklib, config, m3core, libm3 > text, threads > not a gui > This is subjective. The "resource" stuff should probably be here. The port is only going to install one distribution, and that will be the "all". The non-used one is just a waste of building; it will be thrown away. Ports framework repackages and stored in FreeBSD's binary package format. > 7) You didn't ask. But I kinda think we should only have environment > variables > that start CM3 -- CM3 and start CM3_ actually. that sounds logical but as long as they are defined it doesn't really matter to me. > 8) Another large meta point -- this is all clearly imperfect, usually > for lack of time or energy, not because we felt it was better this way. > Though there is the angle I gave -- portability. > Also, autoconf was probably less well established when most of this was > written. > The build/conf world seems still to be somewhat unsettled. if a python script is sufficient for port needs, this issue isn't important to me. So far it seems that it is. > 9) I have a larger unimplemented vision -- the system should be > redistributed as one portable hard to read C or C++ .tar.gz. > This doesn't work today, because "the frontend does layout" -- we > essentially have 6 flavors of C: > We could today make 6 distributions of just C and bootstrap from one of > them, picking the right one. > Most 64bit systems can run 32bit, so strike those as bootstrap? But not > quite, e.g. OpenBSD. DragonFly is also 64-bit only. > I'd like the option for the frontend to defer layout to the backend, and > for the C backend to output string expressions involving sizeof(size_t) > or sizeof(char*). > And, for win32 vs. posix, I'd like quake to compile both, and to > generate a Makefile that picks the right one, or a .sh/.cmd. > Oh, and jmpbuf size is in the generated C. I know how to fix that. > bootstrap specifically could inflate it to the largest known case, which > is quite large. > Or, once we generate C++ for excpetion handling, jmpbuf should go away. no comment. :) John From jay.krell at cornell.edu Wed Jun 3 11:30:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 09:30:44 +0000 Subject: [M3devel] It works! In-Reply-To: <556EBCA3.3090601@marino.st> References: <556700B0.8060500@marino.st>, , , , ,,<556B4BE9.1000902@marino.st>, , , , ,,,,, , , , ,,,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , ,,,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , ,,, , , , , , , , <556C1218.3000103@marino.st> , , , , , <556C16E7.505@marino.st>, , , , ,,,,<556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st>, , <556EBCA3.3090601@marino.st> Message-ID: I believe people typically install to /usr/local/cm3 instead of /usr/local.There is a philosophical problem here in terms of setup file systemlayout. //package and then symlinks from /usr/local/{bin,lib}and removal of a package is just removing its one directory recursivelyand cleaning up any orphaned links vs. files put directly into/usr/local/{bin,lib} and data maintained on the side as to which filecame from which package. The first way gives more leeway to install extra stuff but not linkit into $PATH etc. The second way I believe is more common.It is easy to put together by make DESTDIR=whatever, tar.gz whateverand install is just untar, and uninstall either isn't implemented,or the data on the side is the output of tar tf, I guess. I put it in /cm3 on my systems and $HOME/cm3 where I don'thave that access, or $HOME/cm3. for filesystemsshared across targets -- e.g. the opencsw machines share filesystemacross sparc32, sparc64, x86, amd64. But yes, I understand. Just because it is called "config", does not make it "config".It should be refactored.And will there be some expectation that the "binaries" can beupdated while leaving "config" alone? > currently I have to patch make-dist.py to make it not > tar/gzip. If the script supported the option, I could avoid that patch. Send your patches here? Or fork on github and send pull requests? Another "less-binary" bootstrap option is the C archives.It is text files, but they are all generated and hard to read.I have to get the C backend back to working. :( Since you are going into ports, with an expected install place,and are building it yourself, you might want to mess withrpath, see here: jbook2:config-no-install jay$ pwd/dev2/cm3/m3-sys/cminstall/src/config-no-install book2:config-no-install jay$ grep rpath *ALPHA_OSF: args += [ "-rpath", LIB_USE ]FreeBSD.common: & " -Wl,-rpath,\\$ORIGIN"FreeBSD.common: & " -Wl,-rpath,\\$ORIGIN/../lib "Interix.common: % & " -Wl,-rpath,\\$ORIGIN"Interix.common: % & " -Wl,-rpath,\\$ORIGIN/../lib"Linux.common: & " -Wl,-rpath,\\$ORIGIN"Linux.common: & " -Wl,-rpath,\\$ORIGIN/../lib"NetBSD.common:M3_SHARED_LIB_ARG = "-Wl,-rpath-link,"OpenBSD.common:% & " -Wl,-rpath,\\$ORIGIN" % no $ORIGIN support up to/including 4.7OpenBSD.common:% & " -Wl,-rpath,\\$ORIGIN/../lib" % no $ORIGIN support up to/including 4.7OpenBSD.common:SYSTEM_LIBS{"ODBC"} = ["-Wl,-rpath,/usr/local/lib -L/usr/local/lib -liodbc" ]OpenBSD.common:SYSTEM_LIBS{"POSTGRES95"} = ["-Wl,-rpath -L/usr/local/lib -lpq"]gnuld.common: M3_SHARED_LIB_ARG = "-Wl,-rpath,"gnuld.common: GNU_LD_APPEND = " -Wl,-rpath," & INSTALL_ROOT & "/lib " % non-portablegnuld.common: M3_SHARED_LIB_ARG = "-Wl,-rpath," - Jay > Date: Wed, 3 Jun 2015 10:36:51 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] It works! > > On 6/3/2015 09:18, Jay K wrote: > > 1) "config"'s user configurability is over-implied imho. > > [snip] > > But also, we target not just Unix. > > Yet we have almost the same directory structure on Unix and Windows. > > The difference is that on Windows we put the .dlls in bin. > > It is possible this was misunderstood. > > It's important to release that cm3 will be installed nominally at > /usr/local/bin. On a normal unix system, there could be over 500 > packages all installing there. Things like configuration files (.conf) > are nominally installed at /usr/local/etc/. The config directory is > analogous to these. > > I will do the following in any case: > A) move config from bin to etc// > B) edit cm3.cfg to point to new location > > All I was suggesting is that the script do this for me. It can still > install at bin/ by default, but it would be nice if it could install > elsewhere too. > > > > > Should we use $HOME/etc or $HOME/config? > > Never. > This does not work for packaging. > E.g. prebuild binary packages that are installed (no building required) > > > > > 2) "example" progs > > Yes. Remember how I said I was lazy and made just one, or rather two, > > package sets? That is what you are seeing here. > > Olaf did put more thought/work into this and the .sh files do build a > > few different distribution sets. > > I am fine with what is built. Remember above that cm3 is installed with > 500 other packages. Any program that is not essential and essentially > there as an example could (and my opinion) should be installed in a > place other than bin/. I don't care if it's installed outside of PATH > because it probably won't be called much and absolute path is fine in > this case. > > > 3) I think so. Look for "www" in config? > > 1-3) yes. > > In particular, are you aware of the > > rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? > > > I just remember that the current cm3 in ports, which is split up like I > suggest, had some broken links because I moved these things around. I > can't remember what is actually broken; I'd have to check again. The > point was you can't just move stuff around without changing links, so > the install script would have to do this if it supported alternative > locations. > > > > 4) > > Just command line parsing in make-dist.py. > > Right, but currently I have to patch make-dist.py to make it not > tar/gzip. If the script supported the option, I could avoid that patch. > > > > 5) yes > > 6) I don't understand. > > min is meant to be: > > 6a) a minimal toolset > > You can write hello world. > > cm3, cm3cg, mklib, config, m3core, libm3 > > text, threads > > not a gui > > This is subjective. The "resource" stuff should probably be here. > > > The port is only going to install one distribution, and that will be the > "all". The non-used one is just a waste of building; it will be thrown > away. > > Ports framework repackages and stored in FreeBSD's binary package format. > > > 7) You didn't ask. But I kinda think we should only have environment > > variables > > that start CM3 -- CM3 and start CM3_ actually. > > that sounds logical but as long as they are defined it doesn't really > matter to me. > > > > 8) Another large meta point -- this is all clearly imperfect, usually > > for lack of time or energy, not because we felt it was better this way. > > Though there is the angle I gave -- portability. > > Also, autoconf was probably less well established when most of this was > > written. > > The build/conf world seems still to be somewhat unsettled. > > if a python script is sufficient for port needs, this issue isn't > important to me. So far it seems that it is. > > > 9) I have a larger unimplemented vision -- the system should be > > redistributed as one portable hard to read C or C++ .tar.gz. > > This doesn't work today, because "the frontend does layout" -- we > > essentially have 6 flavors of C: > > We could today make 6 distributions of just C and bootstrap from one of > > them, picking the right one. > > Most 64bit systems can run 32bit, so strike those as bootstrap? But not > > quite, e.g. OpenBSD. > > DragonFly is also 64-bit only. > > > I'd like the option for the frontend to defer layout to the backend, and > > for the C backend to output string expressions involving sizeof(size_t) > > or sizeof(char*). > > And, for win32 vs. posix, I'd like quake to compile both, and to > > generate a Makefile that picks the right one, or a .sh/.cmd. > > Oh, and jmpbuf size is in the generated C. I know how to fix that. > > bootstrap specifically could inflate it to the largest known case, which > > is quite large. > > Or, once we generate C++ for excpetion handling, jmpbuf should go away. > > no comment. :) > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 11:54:50 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 11:54:50 +0200 Subject: [M3devel] It works! In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , , <556C16E7.505@marino.st>, , , , , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st>, , <556EBCA3.3090601@marino.st> Message-ID: <556ECEEA.9040302@marino.st> On 6/3/2015 11:30, Jay K wrote: > I believe people typically install to /usr/local/cm3 instead of /usr/local. > There is a philosophical problem here in terms of setup file system > layout. //package and then symlinks from /usr/local/{bin,lib} > and removal of a package is just removing its one directory recursively > and cleaning up any orphaned links vs. files put directly into > /usr/local/{bin,lib} and data maintained on the side as to which file > came from which package. Well, setting it to /usr/local/cm3 would eliminate name clashes, but it would also move cm3 out of the standard path. That would require every user to adjust their path. If were were talking about 1-3 symlinks in /usr/local/bin then that would be fine, but if it's like 25+ then that could be kind of pointless. Plus on FreeBSD documentation, config, and examples go in standard locations (if possible) and setting to /usr/local/cm3 without modification wouldn't solve that. We aren't worried about orphan files or whatever, the binary package manager cleans up after itself when a package is removed, and won't let two versions of a package be installed simultaneously, so there's no real benefit to having cm3 in a dedicated directory (IMO). > Just because it is called "config", does not make it "config". > It should be refactored. > And will there be some expectation that the "binaries" can be > updated while leaving "config" alone? No. With updates, everything that was installed gets removed. If this is part of an update, everything is reinstalled. There's always integrity there. For BSD setups, having it at etc/modula3/config makes a lot of sense and there's no downside other than it's a different location from releases that you guys may make. > > currently I have to patch make-dist.py to make it not > > tar/gzip. If the script supported the option, I could avoid that patch. > > > Send your patches here? > Or fork on github and send pull requests? They wouldn't be suitable. They would be patches to make the script work meaning lines would be removed. You're thinking about a patch to handle both cases. > Since you are going into ports, with an expected install place, > and are building it yourself, you might want to mess with > rpath, see here: IIRC, the rpath was already fixed with the CM3 portable rpath option. I sort of recall hitting that before and finding $ORIGIN supported. If RPATH is wrong it definitely needs fixing but I think it is a problem that presents quickly (e.g. you couldn't use the product as a bootstrap). John From wagner at elegosoft.com Wed Jun 3 13:11:02 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 3 Jun 2015 13:11:02 +0200 Subject: [M3devel] It works! In-Reply-To: <556ECEEA.9040302@marino.st> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> Message-ID: <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> On Wed, 03 Jun 2015 11:54:50 +0200 John Marino wrote: > On 6/3/2015 11:30, Jay K wrote: > > I believe people typically install to /usr/local/cm3 instead of /usr/local. > > There is a philosophical problem here in terms of setup file system > > layout. //package and then symlinks from /usr/local/{bin,lib} > > and removal of a package is just removing its one directory recursively > > and cleaning up any orphaned links vs. files put directly into > > /usr/local/{bin,lib} and data maintained on the side as to which file > > came from which package. > > Well, setting it to /usr/local/cm3 would eliminate name clashes, but it > would also move cm3 out of the standard path. That would require every > user to adjust their path. > > If were were talking about 1-3 symlinks in /usr/local/bin then that > would be fine, but if it's like 25+ then that could be kind of pointless. > > Plus on FreeBSD documentation, config, and examples go in standard > locations (if possible) and setting to /usr/local/cm3 without > modification wouldn't solve that. > > We aren't worried about orphan files or whatever, the binary package > manager cleans up after itself when a package is removed, and won't let > two versions of a package be installed simultaneously, so there's no > real benefit to having cm3 in a dedicated directory (IMO). > > > Just because it is called "config", does not make it "config". > > It should be refactored. > > And will there be some expectation that the "binaries" can be > > updated while leaving "config" alone? > > No. With updates, everything that was installed gets removed. If this > is part of an update, everything is reinstalled. There's always > integrity there. For BSD setups, having it at etc/modula3/config makes > a lot of sense and there's no downside other than it's a different > location from releases that you guys may make. It doesn't really fit, but never mind. What is different about M3 is that it contains both the base development system and dozens of utility and application packages, and every user is supposed to be able to update and install packages (only it's called shipping, not installation). Theoretically, you could package each M3 package in its own BSD package; I even think there was one (historical) distribution of PM3 that did exactly that. That doesn't address the problem, as each user needs to become root to ship a package. As I've written in another m3 thread yesterday, what would really be needed is a multi-level package pool hierarchy and an integrated m3 builder that copes with that and sets of packages. But even if we had that, I'm not sure how it would fit into the standard Unix file system hierarchy layout: put the user-managed package pools under /var/local/lib? As the package systems vary significantly between all the supported platforms of M3, the concepts of M3 packages and OS packages have never been unified. And of course the M3 project never had enough personal resources to provide OS specific packages for even a small subset of the target platforms. Perhaps a pragmatical way to handle this would be to package and install a minimal system only containing the builder, compiler and base libraries as an OS package in an OS-specific way (including documentation and all) and add support for the standard M3 package hierarchy under /usr/local/cm3 (group-writable by an m3 group). It would be a compromise, but perhaps a better one than installing the complete CM3 distribution as one big package read-only for the ordinary user. The CM3 package pool under /usr/local/cm3 could even be integrated with the Github repository and allow easy checkout of all M3 packages there and even pushing new branches and packages. The builder could locate, checkout and install missing imported packages, have an option for creating a new package template and a standard way to publish this on Gibhub. This might help to lower the threshold for new users to become active in the community and share their code. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From adacore at marino.st Wed Jun 3 13:46:07 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 13:46:07 +0200 Subject: [M3devel] It works! In-Reply-To: <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> Message-ID: <556EE8FF.3060502@marino.st> On 6/3/2015 13:11, Olaf Wagner wrote: > What is different about M3 is that it contains both the base development > system and dozens of utility and application packages, and every user > is supposed to be able to update and install packages (only it's > called shipping, not installation). If I just install the "all" distribution, as built, in /usr/local/cm3 (with the possible exception of shifting www to standard docs location): 1) It is a read-only area (ideally) 2) It would require all users to set /usr/local/cm3 in PATH. This is not really that big of a deal. It's not the first package to require something like that, plus, frankly, the m3 community is not large and such a requirement could be imposed without a lot of backlash. Maybe this is the best solution after all (and again it makes the port makefile simpler) > Theoretically, you could package each M3 package in its own BSD > package; I even think there was one (historical) distribution of PM3 > that did exactly that. That doesn't address the problem, as each user > needs to become root to ship a package. Breaking each into it's own package is what should happen, but that requires work on my part that I'm not able to give, so everyone has to deal with the giant port. If m3 regains enough popularity to justify splitting into individual packages then we can revisit the decision. > As I've written in another m3 thread yesterday, what would really > be needed is a multi-level package pool hierarchy and an integrated > m3 builder that copes with that and sets of packages. But even if we had > that, I'm not sure how it would fit into the standard Unix file system > hierarchy layout: put the user-managed package pools under /var/local/lib? At least for ports, packages either go in root's areas (e.g. /usr/local) or user-accessible areas but not mixed. One can install M3 in a user's area and do what they want. But a generic user can't install to standard areas like /usr/local and most /var areas. > As the package systems vary significantly between all the supported > platforms of M3, the concepts of M3 packages and OS packages have > never been unified. And of course the M3 project never had enough > personal resources to provide OS specific packages for even a small > subset of the target platforms. Pragmatically, you want the individual platforms to package it for you. Less work for you and it's probably done more correctly. E.g. m3 is self-hosting on FreeBSD so there's no real need to provide FreeBSD support now (with the possible exception of a statically linked "min" distribution) Ideally, similar support occurs on Debian, Fedora, Arch, macports, etc. John From adacore at marino.st Wed Jun 3 13:57:38 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 13:57:38 +0200 Subject: [M3devel] Are all 5 gcc branches used? Message-ID: <556EEBB2.30809@marino.st> I see 5 gcc branches in m3-sys/m3cc. I've seen gcc-4.7 in use and I can guess gcc-apple is still used, but are gcc, gcc-4.5, and gcc-4.6 branches obsolete? There reason why it matters is that I'm using github's built-in "create tarball from top of repo" capability as the digest-confirmed source distribution file. All the extra gcc branches are causing the compressed tarball to be pretty big. Currently it's at 150Mb. If some of these gcc branches are unused and will never again be used, I might suggest they are pruned. It's in the repository so they could be retrieved with the right hash, and it would dramatically reduce the size of the archive tarball. If all 5 versions of gcc are still used then that's a horse of a different color. John From jay.krell at cornell.edu Wed Jun 3 18:36:17 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 3 Jun 2015 09:36:17 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: > Other than cross-compile from 32 to 64 bit what purpose does this serve? > > Sent from my iPad > > On Jun 2, 2015, at 9:58 PM, Jay K wrote: > >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 >> index fa72589..37bf238 100644 >> --- a/m3-libs/m3core/src/text/TextLiteral.i3 >> +++ b/m3-libs/m3core/src/text/TextLiteral.i3 >> @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; >> IMPORT RTHooks, TextClass; >> >> CONST >> - (* DIV BITSIZE should not be here! *) >> +(* When cm3 uses TInt and when pickle special cases this, it should be approx: >> + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) >> +(* This fails for 32bit host and 64bit target: >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) >> +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) >> + dependent. *) >> + (* DIV BITSIZE should not be here -- compiler limitation due to >> + non-use of TInt. *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; >> (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) >> (* Do not adjust this for INTEGER size. It makes T have different >> fingerprints on 32- and 64-bit machines, which undermines pickling/ >> >> >> Yes I know TEXT pickles will be broken. >> I propose that unpickle should special case a few values here. >> Or, isn't the brand supposed to help? >> Or, can we encode this in a better way, w/o an upper bound? >> I know if you put in 0..0 or 0..-1 you incur range violations at runtime. >> Which makes me wonder -- are TEXTs unsafe? >> >> Should we support a syntax something like: >> >> cnt : INTEGER; >> buf : ARRAY [0..cnt - 1] OF Byte; >> ? >> >> Thanks, >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 18:39:46 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 3 Jun 2015 09:39:46 -0700 Subject: [M3devel] It works! In-Reply-To: <556EE8FF.3060502@marino.st> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> <556EE8FF.3060502@marino.st> Message-ID: <5649AA84-7127-49E1-9F80-7E3F425A2DE5@gmail.com> I am sympathetic to "path pollution" concerns even if in usr/local/cm3. We can move some stuff to demobin or samplebin if agreement on uselessness. And/or prefix every bin & lib with m3 or cm3. We should also rev our so versions? & and install version-named cm3 & cm3cg? - Jay On Jun 3, 2015, at 4:46 AM, John Marino wrote: > On 6/3/2015 13:11, Olaf Wagner wrote: >> What is different about M3 is that it contains both the base development >> system and dozens of utility and application packages, and every user >> is supposed to be able to update and install packages (only it's >> called shipping, not installation). > > If I just install the "all" distribution, as built, in /usr/local/cm3 > (with the possible exception of shifting www to standard docs location): > > 1) It is a read-only area (ideally) > 2) It would require all users to set /usr/local/cm3 in PATH. > > This is not really that big of a deal. It's not the first package to > require something like that, plus, frankly, the m3 community is not > large and such a requirement could be imposed without a lot of backlash. > > Maybe this is the best solution after all (and again it makes the port > makefile simpler) > > >> Theoretically, you could package each M3 package in its own BSD >> package; I even think there was one (historical) distribution of PM3 >> that did exactly that. That doesn't address the problem, as each user >> needs to become root to ship a package. > > Breaking each into it's own package is what should happen, but that > requires work on my part that I'm not able to give, so everyone has to > deal with the giant port. If m3 regains enough popularity to justify > splitting into individual packages then we can revisit the decision. > >> As I've written in another m3 thread yesterday, what would really >> be needed is a multi-level package pool hierarchy and an integrated >> m3 builder that copes with that and sets of packages. But even if we had >> that, I'm not sure how it would fit into the standard Unix file system >> hierarchy layout: put the user-managed package pools under /var/local/lib? > > At least for ports, packages either go in root's areas (e.g. /usr/local) > or user-accessible areas but not mixed. One can install M3 in a user's > area and do what they want. But a generic user can't install to > standard areas like /usr/local and most /var areas. > > >> As the package systems vary significantly between all the supported >> platforms of M3, the concepts of M3 packages and OS packages have >> never been unified. And of course the M3 project never had enough >> personal resources to provide OS specific packages for even a small >> subset of the target platforms. > > Pragmatically, you want the individual platforms to package it for you. > Less work for you and it's probably done more correctly. E.g. m3 is > self-hosting on FreeBSD so there's no real need to provide FreeBSD > support now (with the possible exception of a statically linked "min" > distribution) > > Ideally, similar support occurs on Debian, Fedora, Arch, macports, etc. > > John > > From jay.krell at cornell.edu Wed Jun 3 19:49:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 17:49:07 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> Message-ID: I don't have any, but should we have such artificial limits? I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." And even that isn't right -- 4bg is often not lenty for a 32bit host. File sizes exceed that long ago.But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. Given that a blueray movie is an array of bytes, is a text inappropriate? The actual diff presented just adjusts the limit by 8 bytes. Ok? Or must preserve pickles and can't do this w/o some other change? This isn't a more general problem? Or usually pickles are 32/64-compatible, but this is a most unusual case? Thanks, - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 13:24:36 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Why would you ever want a TEXT literal that is so large?By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface.A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. On Jun 3, 2015, at 12:36 PM, Jay wrote:I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: Other than cross-compile from 32 to 64 bit what purpose does this serve? Sent from my iPad On Jun 2, 2015, at 9:58 PM, Jay K wrote: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 20:04:04 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 18:04:04 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556EEBB2.30809@marino.st> References: <556EEBB2.30809@marino.st> Message-ID: They likely aren't all used.I'd like to make none of them used, but that is another matter. Look in src/m3makefile. I think OpenBSD targets might be back on gcc-4.2.There is flexibility in there, in m3makefile and "parse.c", but maybe at too high cost. The problem, you can imagine, is that Apple has as fork, OpenBSD has a fork, we have a fork, and there is mainline.Ideally everyone would be in mainline and nobody would have any forks. As we/I move from major version to version, I generally make a new fork. I have a certain paranoia and laziness when debugging -- "does it work with the old one" -- "how easy is to reconstruct the old one? Oh, cool, it is still right there, I don't have to figure out how to go backwards in source control, and I can easily have them both side by side". I realize source control could help me here. It also allows for "staging", i.e. I can bring in new version, test some targets, but leave the other targets alone, waiting for myself or others to test them later. Or, decide they are little enough used just move them forward w/o testing. If gcc obsoletes targets we want to keep, we could keep the old versions. Not super useful given our usage levels. We could maybe also minimize our changes and distribute patches only.Sometimes maybe I went overboard with my changes. Or we could ditch gcc entirely and use the C backend and/or LLVM, hopefully both unpatched. :) Try xz instead of gzip, maybe it halves the size? - Jay > Date: Wed, 3 Jun 2015 13:57:38 +0200 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Are all 5 gcc branches used? > > I see 5 gcc branches in m3-sys/m3cc. I've seen gcc-4.7 in use and I can > guess gcc-apple is still used, but are gcc, gcc-4.5, and gcc-4.6 > branches obsolete? > > There reason why it matters is that I'm using github's built-in "create > tarball from top of repo" capability as the digest-confirmed source > distribution file. All the extra gcc branches are causing the > compressed tarball to be pretty big. Currently it's at 150Mb. > > If some of these gcc branches are unused and will never again be used, I > might suggest they are pruned. It's in the repository so they could be > retrieved with the right hash, and it would dramatically reduce the size > of the archive tarball. > > If all 5 versions of gcc are still used then that's a horse of a > different color. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 20:10:07 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 20:10:07 +0200 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st> Message-ID: <556F42FF.9000106@marino.st> On 6/3/2015 20:04, Jay K wrote: > If gcc obsoletes targets we want to keep, we could keep the old > versions. Not super useful given our usage levels. It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not obsolete by that definition, so all 5 of these are obsolete. I would definitely encourage to prune as many of these as possible. I could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" based on the base compiler, why are you assuming it must be built by base compiler? By definition, the base compiler is only required to build base. There are much newer and well maintained versions of gcc in openbsd ports tree. There's no reason a ports compiler couldn't be used (I assume this is actually common). > Try xz instead of gzip, maybe it halves the size? I can't influence github's API. It is what it is. John From hendrik at topoi.pooq.com Wed Jun 3 21:07:46 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 3 Jun 2015 15:07:46 -0400 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st> Message-ID: <20150603190745.GA11809@topoi.pooq.com> On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > hopefully both unpatched. :) Weren't you working on a C backend for Modula a year or two ago? What happened to it? Is it available for use? If so, where and how? -- hendrik From rodney_bates at lcwb.coop Wed Jun 3 21:23:07 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 03 Jun 2015 14:23:07 -0500 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: References: Message-ID: <556F541B.5080207@lcwb.coop> On 06/02/2015 09:51 PM, Jay K wrote: > Ideally: > 1 32bit host could target 64bit target > 2 32bit texts could be as large as the address space/OS allows. > 3 ditto 64bit texts -- larger than 4GB > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > Currently only #4 is true. > TEXTs for any target can only be about 500MB. > > > To fix 2 and 3 requires m3front to use TInt more. > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > Can we special case somehow? > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > Should/can we introduce one? > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > index fa72589..4d16a44 100644 > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > CONST > (* DIV BITSIZE should not be here! *) > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; I'd say go ahead and change this. We really do want it to cross-compile sooner than anybody will be able to put TInt in all over the place. I did not build a cross compiler, but did what I think should be a realistic simulation of it by changing the cnt:INTEGER field to two INTEGER fields and compiling with a 32-bit compiler. That would be the same size as a 64-bit INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to compile, with bit size having space for 31 more bits before it overflows. This byte count of buf is a multiple of 4, which we want for unicode-range WIDECHAR. Making it -10 overflows, presumably because the additional byte is padded out to a multiple of 32 bits. I would have expected that somewhere, a size including the one-word heap header would be computed, but these are never actually heap allocated, and it seems to work. But to hedge against things, I suggest using -19. This would allow space to count it, cross-compiling 32->64. This will allow a text literal of 268435436 ISO characters, and a wide text literal of 1/4 that, which is not likely to be a serious limitation, considering that it all has to be on a single line of source code. As for pickles, the horse is already out of the barn on that, as pickles could already have been written with the current fingerprint, and we can't change the type in any way without the fingerprint changing. It is not hard to add recognition of the old fingerprint to pickles. Right off hand, I don't remember the process for finding actual fingerprint values, but I've done it before and can rediscover it. > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > (* Do not adjust this for INTEGER size. It makes T have different > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > otherwise we have: > > > new source -> compiling TextLiteral.i3 > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > 1 error encountered > > > Ok? > - Jay -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Jun 3 21:29:24 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 03 Jun 2015 14:29:24 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <556F5594.7070701@lcwb.coop> On 06/02/2015 08:58 PM, Jay K wrote: > We cannot cross from 32bit host to 64bit target. > > > Ok to commit this? > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > index fa72589..37bf238 100644 > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; > IMPORT RTHooks, TextClass; > CONST > - (* DIV BITSIZE should not be here! *) > +(* When cm3 uses TInt and when pickle special cases this, it should be approx: > + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) > +(* This fails for 32bit host and 64bit target: > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) > +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) > + dependent. *) > + (* DIV BITSIZE should not be here -- compiler limitation due to > + non-use of TInt. *) > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > (* Do not adjust this for INTEGER size. It makes T have different > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > Yes I know TEXT pickles will be broken. > I propose that unpickle should special case a few values here. > Or, isn't the brand supposed to help? > Or, can we encode this in a better way, w/o an upper bound? > I know if you put in 0..0 or 0..-1 you incur range violations at runtime. > Which makes me wonder -- are TEXTs unsafe? > > Should we support a syntax something like: > > cnt : INTEGER; > buf : ARRAY [0..cnt - 1] OF Byte; > ? > Can't do this. Non-open arrays must have static constant bounds, which the above does not. Open arrays have extra dope and are accessed through two extra pointers, which we really wouldn't want for text literals. TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that anything else in UNSAFE code. The interface is UNSAFE, which means safe code can't IMPORT it, and thus can't see the declaration. Not very likely that anybody other than the runtime, compiler-generated code, and debugger (written in C anyway) would want to mess with it. > Thanks, > - Jay -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 4 00:55:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 22:55:00 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556F42FF.9000106@marino.st> References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: > obsolete I meant, gcc 4.9 might not target Alpha/OSF, but gcc 4.5 might. > OpenBSD gcc 4.2. True -- we could take the OpenBSD port of gcc 4.7 or such and apply those patches. In fact... 1) most of the patches are to the driver, that we don't use Things like building with #define __OpenBSD__ and making size_t == unsigned long or unsigned int. 2) Maybe I'm doing this. I recall checking patch files and applying them either at build time or commiting those. 3) Really, you know, from a compiler backend point of view, OpenBSD, NetBSD, FreeBSD, all the same thing -- same ABI, same x86 instructions, same object file format. Various targets collapse down to fewer. So we don't always need the patches. But then something like ARM_DARWIN, which I never got fully working, those might be some more serious patches. 4.2 might also be Apple or Interix related. I'll have to look. We could probably also getting away with such things as: OpenBSD, Interix, ARM_DARWIN: C backend only, if even that I do have to restore it to working. It was totally working. Anyway, point taken/reminded -- we should see about pruning. I just get nervous, you know, I'm not a CVS or git expert -- how do I get back the old stuff? > xz vs. gzip Does the ports system now download from github? Cool. gzip is much faster for compression. - Jay > Date: Wed, 3 Jun 2015 20:10:07 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On 6/3/2015 20:04, Jay K wrote: > > If gcc obsoletes targets we want to keep, we could keep the old > > versions. Not super useful given our usage levels. > > It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not > obsolete by that definition, so all 5 of these are obsolete. > > I would definitely encourage to prune as many of these as possible. I > could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" > based on the base compiler, why are you assuming it must be built by > base compiler? > > By definition, the base compiler is only required to build base. There > are much newer and well maintained versions of gcc in openbsd ports > tree. There's no reason a ports compiler couldn't be used (I assume > this is actually common). > > > Try xz instead of gzip, maybe it halves the size? > > I can't influence github's API. It is what it is. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 00:58:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 22:58:50 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <20150603190745.GA11809@topoi.pooq.com> References: <556EEBB2.30809@marino.st>, , <20150603190745.GA11809@topoi.pooq.com> Message-ID: It was totally working. I built the whole Darwin x86/amd64 system, ran gui apps.Got AMD64_NT working.Built all four variations of Solaris I think.Maybe cross bootstrapped but didn't c_compile other systems.I386_NT was probably working. And now if you try it there are assertion failures, like about right shiftinga signed type. Some errors about duplicate typedefs. It has bit-rotted slightly. Maybe with the Unicode work? Anyway, I'll repair it. It really was fully working.Debugging was getting better, better than non-m3gdb systems.There is still a uniquifier appended to every local.In case you have; int a;{ int a;{ int a;}}} we give youint a_1;int a_2;int a_3; I need to fix it to only uniqueify in the presence of duplicates. Globals should be made debuggable instead of just being an array of bytes.Other than globals, structs have fields.TEXTs aren't viewable I think. But for systems like Darwin and NT and HP-UX that don't support m3gdb, debugging is already better.Build times are noticeably regressed. - Jay > Date: Wed, 3 Jun 2015 15:07:46 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > > hopefully both unpatched. :) > > Weren't you working on a C backend for Modula a year or two ago? > What happened to it? Is it available for use? If so, where and how? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:04:43 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:04:43 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st>, , , , <20150603190745.GA11809@topoi.pooq.com>, Message-ID: Once it is working, the way you use it is one of: ./do-cm3-all.py c buildship or change M3_BACKEND_MODE to "C" in the config file. and it handles nested functions even... You can bootstrap with./boot1.py c which gives you a directory of c files and a makefile.cd there and make or nmake. But let me fix it first. It got broken recently. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Wed, 3 Jun 2015 22:58:50 +0000 Subject: Re: [M3devel] Are all 5 gcc branches used? It was totally working. I built the whole Darwin x86/amd64 system, ran gui apps.Got AMD64_NT working.Built all four variations of Solaris I think.Maybe cross bootstrapped but didn't c_compile other systems.I386_NT was probably working. And now if you try it there are assertion failures, like about right shiftinga signed type. Some errors about duplicate typedefs. It has bit-rotted slightly. Maybe with the Unicode work? Anyway, I'll repair it. It really was fully working.Debugging was getting better, better than non-m3gdb systems.There is still a uniquifier appended to every local.In case you have; int a;{ int a;{ int a;}}} we give youint a_1;int a_2;int a_3; I need to fix it to only uniqueify in the presence of duplicates. Globals should be made debuggable instead of just being an array of bytes.Other than globals, structs have fields.TEXTs aren't viewable I think. But for systems like Darwin and NT and HP-UX that don't support m3gdb, debugging is already better.Build times are noticeably regressed. - Jay > Date: Wed, 3 Jun 2015 15:07:46 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > > hopefully both unpatched. :) > > Weren't you working on a C backend for Modula a year or two ago? > What happened to it? Is it available for use? If so, where and how? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:14:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:14:49 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> Message-ID: sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched.An array of chars? Probably has same problem. Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 14:07:02 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu It?s only TEXT literals that are limited here. As in, what appears in a source program. On Jun 3, 2015, at 1:49 PM, Jay K wrote:I don't have any, but should we have such artificial limits? I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." And even that isn't right -- 4bg is often not lenty for a 32bit host. File sizes exceed that long ago.But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. Given that a blueray movie is an array of bytes, is a text inappropriate? The actual diff presented just adjusts the limit by 8 bytes. Ok? Or must preserve pickles and can't do this w/o some other change? This isn't a more general problem? Or usually pickles are 32/64-compatible, but this is a most unusual case? Thanks, - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 13:24:36 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Why would you ever want a TEXT literal that is so large?By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface.A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. On Jun 3, 2015, at 12:36 PM, Jay wrote:I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: Other than cross-compile from 32 to 64 bit what purpose does this serve? Sent from my iPad On Jun 2, 2015, at 9:58 PM, Jay K wrote: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:18:11 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:18:11 +0000 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: <556F541B.5080207@lcwb.coop> References: , <556F541B.5080207@lcwb.coop> Message-ID: Should we just be a little lax and say minus 64?Or pick some other number like 256MB? Can we declare there is no bound? Thank you, - Jay > Date: Wed, 3 Jun 2015 14:23:07 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) > > > > On 06/02/2015 09:51 PM, Jay K wrote: > > Ideally: > > 1 32bit host could target 64bit target > > 2 32bit texts could be as large as the address space/OS allows. > > 3 ditto 64bit texts -- larger than 4GB > > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > > Currently only #4 is true. > > TEXTs for any target can only be about 500MB. > > > > > > To fix 2 and 3 requires m3front to use TInt more. > > > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > > Can we special case somehow? > > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > > Should/can we introduce one? > > > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > > index fa72589..4d16a44 100644 > > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > > CONST > > (* DIV BITSIZE should not be here! *) > > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > > I'd say go ahead and change this. We really do want it to cross-compile > sooner than anybody will be able to put TInt in all over the place. > > I did not build a cross compiler, but did what I think should be a realistic > simulation of it by changing the cnt:INTEGER field to two INTEGER fields and > compiling with a 32-bit compiler. That would be the same size as a 64-bit > INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to > compile, with bit size having space for 31 more bits before it overflows. > This byte count of buf is a multiple of 4, which we want for unicode-range > WIDECHAR. Making it -10 overflows, presumably because the additional byte is > padded out to a multiple of 32 bits. > > I would have expected that somewhere, a size including the one-word heap header > would be computed, but these are never actually heap allocated, and it seems to > work. But to hedge against things, I suggest using -19. This would allow space > to count it, cross-compiling 32->64. > > This will allow a text literal of 268435436 ISO characters, and a wide text literal > of 1/4 that, which is not likely to be a serious limitation, considering that it > all has to be on a single line of source code. > > As for pickles, the horse is already out of the barn on that, as pickles could > already have been written with the current fingerprint, and we can't change the > type in any way without the fingerprint changing. It is not hard to add > recognition of the old fingerprint to pickles. Right off hand, I don't remember > the process for finding actual fingerprint values, but I've done it before and > can rediscover it. > > > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > > (* Do not adjust this for INTEGER size. It makes T have different > > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > > > > > otherwise we have: > > > > > > new source -> compiling TextLiteral.i3 > > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > > 1 error encountered > > > > > > Ok? > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Thu Jun 4 16:01:37 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Thu, 04 Jun 2015 16:01:37 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <55705A41.7070503@elstel.org> Am 03.06.15 um 03:58 schrieb Jay K: > We cannot cross from 32bit host to 64bit target. > > > Ok to commit this? > > Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. To me this is just another argument for a language specified value range of INTEGER. It makes certain things easier. My suggestion was: TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) BITS 64 FOR INTEGER would only work on x64 systems. The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same tends to be much slower). Regards, Elmar > Can't do this. Non-open arrays must have static constant bounds, > which the above does not. Open arrays have extra dope and are accessed > through two extra pointers, which we really wouldn't want for text > literals. > > TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that > anything > else in UNSAFE code. The interface is UNSAFE, which means safe code > can't IMPORT > it, and thus can't see the declaration. Not very likely that anybody > other than > the runtime, compiler-generated code, and debugger (written in C > anyway) would > want to mess with it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 4 16:07:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 09:07:48 -0500 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: References: , <556F541B.5080207@lcwb.coop> Message-ID: <55705BB4.4090408@lcwb.coop> On 06/03/2015 06:18 PM, Jay K wrote: > Should we just be a little lax and say minus 64? > Or pick some other number like 256MB? > Sure, just to allow for unexpected changes. My principle here was to make it as large as possible, subject to accommodating both native word sizes, in pickling, compiling (including cross compiling), etc. Plus, if it changes, then pickles need to be adapted, so I hope to get it to something we can keep. Do subtract a number that is one less than a multiple of 4, so MaxBytes comes out a multiple of 4. This is pretty confusing. What is really wanted is MaxBytes = (2^31) DIV BITSIZE (Byte) - , but I'm not sure compile-time DIV for this overflow case is well-defined, so 16_7FFFFFFF DIV BITSIZE(Byte) + 1 gives the max multiple of 4 bytes whose bit size is <= LAST(Int32). Then subtract another multiple of 4. So maybe 16_7FFFFFFF DIV BITSIZE(Byte) + 1 - 64 is easier to understand. > Can we declare there is no bound? > No, we're writing code that always ignores the type's bound anyway and uses the cnt field instead. We just want the type's bound as high as possible, to minimize the coding trouble of avoiding the compiler's checks. This is just the kind of low-level coding that the language's safe subset is designed to disallow. > Thank you, > - Jay > > > > Date: Wed, 3 Jun 2015 14:23:07 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) > > > > > > > > On 06/02/2015 09:51 PM, Jay K wrote: > > > Ideally: > > > 1 32bit host could target 64bit target > > > 2 32bit texts could be as large as the address space/OS allows. > > > 3 ditto 64bit texts -- larger than 4GB > > > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > > > Currently only #4 is true. > > > TEXTs for any target can only be about 500MB. > > > > > > > > > To fix 2 and 3 requires m3front to use TInt more. > > > > > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > > > Can we special case somehow? > > > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > > > Should/can we introduce one? > > > > > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > > > index fa72589..4d16a44 100644 > > > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > > > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > > > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > > > CONST > > > (* DIV BITSIZE should not be here! *) > > > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > > > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > > > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > > > > I'd say go ahead and change this. We really do want it to cross-compile > > sooner than anybody will be able to put TInt in all over the place. > > > > I did not build a cross compiler, but did what I think should be a realistic > > simulation of it by changing the cnt:INTEGER field to two INTEGER fields and > > compiling with a 32-bit compiler. That would be the same size as a 64-bit > > INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to > > compile, with bit size having space for 31 more bits before it overflows. > > This byte count of buf is a multiple of 4, which we want for unicode-range > > WIDECHAR. Making it -10 overflows, presumably because the additional byte is > > padded out to a multiple of 32 bits. > > > > I would have expected that somewhere, a size including the one-word heap header > > would be computed, but these are never actually heap allocated, and it seems to > > work. But to hedge against things, I suggest using -19. This would allow space > > to count it, cross-compiling 32->64. > > > > This will allow a text literal of 268435436 ISO characters, and a wide text literal > > of 1/4 that, which is not likely to be a serious limitation, considering that it > > all has to be on a single line of source code. > > > > As for pickles, the horse is already out of the barn on that, as pickles could > > already have been written with the current fingerprint, and we can't change the > > type in any way without the fingerprint changing. It is not hard to add > > recognition of the old fingerprint to pickles. Right off hand, I don't remember > > the process for finding actual fingerprint values, but I've done it before and > > can rediscover it. > > > > > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > > > (* Do not adjust this for INTEGER size. It makes T have different > > > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > > > > > > > > > otherwise we have: > > > > > > > > > new source -> compiling TextLiteral.i3 > > > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > > > 1 error encountered > > > > > > > > > Ok? > > > - Jay > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Jun 4 16:37:06 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 09:37:06 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> Message-ID: <55706292.9060608@lcwb.coop> On 06/03/2015 06:14 PM, Jay K wrote: > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > An array of chars? Probably has same problem. > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > - Jay > I'm OK with doing it, but I thought you had been talking about using TInt. Would that be better? It would be more general. But LONGINT would be faster at compile time, and less work, since the arithmetic operators would not need to be changed to TInt function calls. There might be quite a lot of those I guess the compiler would be of about equal help in finding missed places to be changed, either way. I just checked, and LONGINT is in the release compiler, contrary to my sense of relative history, so there would be no bootstrap barrier. Either would allow a 32-bit hosted compiler (cross- or native-) to handle types whose byte-count approached 2^31, instead of just 2^23. With a 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, whereas LONGINT would only go to 2^55. Do we really care? > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > From: hosking at purdue.edu > Date: Wed, 3 Jun 2015 14:07:02 -0400 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 4 17:08:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:08:38 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55706292.9060608@lcwb.coop> References: , , , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, , <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , , , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, , <55706292.9060608@lcwb.coop> Message-ID: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:41:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:41:47 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55705A41.7070503@elstel.org> References: , <55705A41.7070503@elstel.org> Message-ID: Do you want 64bit targets to be "better" or "the same"?What if I say the Modula-3 equivalent of, three programs: 1 printf("%d", (int)sizeof(void*)); 2 printf("%d", (int)sizeof(size_t)); 3 printf("%d", (int)sizeof(int)); One could argue in your favor here.The user of void* and size_t is expecting "better".The user of int is expecting "same". Modula-3 could have two integer types, builtin, short names.Or, today, it offers all 4 sizes, you just have to go slightly out of your way. You can say: TYPE int = Ctypes.int; or maybe BasicCtypes.int. and go about your business using int instead of INTEGER andget the compability you desire. Perhaps this should be: int = IntegerTypes.INT32? to not be be C-related? - Jay Date: Thu, 4 Jun 2015 16:01:37 +0200 From: estellnb at elstel.org To: jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem Am 03.06.15 um 03:58 schrieb Jay K: We cannot cross from 32bit host to 64bit target. Ok to commit this? Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. To me this is just another argument for a language specified value range of INTEGER. It makes certain things easier. My suggestion was: TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) BITS 64 FOR INTEGER would only work on x64 systems. The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same tends to be much slower). Regards, Elmar Can't do this. Non-open arrays must have static constant bounds, which the above does not. Open arrays have extra dope and are accessed through two extra pointers, which we really wouldn't want for text literals. TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that anything else in UNSAFE code. The interface is UNSAFE, which means safe code can't IMPORT it, and thus can't see the declaration. Not very likely that anybody other than the runtime, compiler-generated code, and debugger (written in C anyway) would want to mess with it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:45:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:45:00 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, <55706292.9060608@lcwb.coop>, , <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: How strongly preferred and why? good naturedly: 1) to cause me pain so that might give up? 2) to take more time so I don't muck with other stuff? 3) for compatibility with older releases? 4) for easier extension in future? 5) to cause me pain so I whine more about wanting operator overloading? I started this once and it going to be a pain. LONGINT is probably much easier. Maybe I have more patience now. Extension in the future could be addition of INTEGER128 to the language. :) And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128and convert frontend to use it. Hypothetically. Perhaps before that happenswe'll have operator overloading. :) - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 11:32:48 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu Again, TInt preferred. Sent from my iPhone On Jun 4, 2015, at 11:08 AM, Jay K wrote: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:54:51 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 4 Jun 2015 08:54:51 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <3A400392-6754-445E-B64F-7E8F1F47985D@purdue.edu> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <3A400392-6754-445E-B64F-7E8F1F47985D@purdue.edu> Message-ID: <119B30EC-1C76-41FB-BCBF-378E287479EF@gmail.com> > I don?t know what the static limit is on an array literal. I suspect all sizes and offsets are limited to bit counts stored INTEGER. So any field or type can be 512mb-1 max on 32bit hosted compiler and 2^56-1 for 64bit hosted compiler. How about a syntax that omits the end of the range? Type t = array [0..] of char? - Jay On Jun 3, 2015, at 5:03 PM, Antony Hosking wrote: > It would not make sense to store an entire blueray movie as a literal?better to structure it. > I don?t know what the static limit is on an array literal. > >> On Jun 3, 2015, at 7:14 PM, Jay K wrote: >> >> sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. >> An array of chars? Probably has same problem. >> >> >> Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? >> >> - Jay >> >> >> >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Wed, 3 Jun 2015 14:07:02 -0400 >> CC: m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> It?s only TEXT literals that are limited here. As in, what appears in a source program. >> >> On Jun 3, 2015, at 1:49 PM, Jay K wrote: >> >> I don't have any, but should we have such artificial limits? >> I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". >> Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." >> And even that isn't right -- 4bg is often not lenty for a 32bit host. >> File sizes exceed that long ago. >> But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. >> >> >> Given that a blueray movie is an array of bytes, is a text inappropriate? >> >> >> The actual diff presented just adjusts the limit by 8 bytes. >> Ok? >> Or must preserve pickles and can't do this w/o some other change? >> This isn't a more general problem? Or usually pickles >> are 32/64-compatible, but this is a most unusual case? >> >> >> Thanks, >> - Jay >> >> >> >> >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Wed, 3 Jun 2015 13:24:36 -0400 >> CC: m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> Why would you ever want a TEXT literal that is so large? >> By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface. >> A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. >> >> On Jun 3, 2015, at 12:36 PM, Jay wrote: >> >> I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. >> >> - Jay >> >> On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: >> >> Other than cross-compile from 32 to 64 bit what purpose does this serve? >> >> Sent from my iPad >> >> On Jun 2, 2015, at 9:58 PM, Jay K wrote: >> >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 >> index fa72589..37bf238 100644 >> --- a/m3-libs/m3core/src/text/TextLiteral.i3 >> +++ b/m3-libs/m3core/src/text/TextLiteral.i3 >> @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; >> IMPORT RTHooks, TextClass; >> >> CONST >> - (* DIV BITSIZE should not be here! *) >> +(* When cm3 uses TInt and when pickle special cases this, it should be approx: >> + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) >> +(* This fails for 32bit host and 64bit target: >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) >> +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) >> + dependent. *) >> + (* DIV BITSIZE should not be here -- compiler limitation due to >> + non-use of TInt. *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; >> (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) >> (* Do not adjust this for INTEGER size. It makes T have different >> fingerprints on 32- and 64-bit machines, which undermines pickling/ >> >> >> Yes I know TEXT pickles will be broken. >> I propose that unpickle should special case a few values here. >> Or, isn't the brand supposed to help? >> Or, can we encode this in a better way, w/o an upper bound? >> I know if you put in 0..0 or 0..-1 you incur range violations at runtime. >> Which makes me wonder -- are TEXTs unsafe? >> >> Should we support a syntax something like: >> >> cnt : INTEGER; >> buf : ARRAY [0..cnt - 1] OF Byte; >> ? >> >> Thanks, >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 4 18:35:07 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 11:35:07 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55705A41.7070503@elstel.org> References: <55705A41.7070503@elstel.org> Message-ID: <55707E3B.7020601@lcwb.coop> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: > Am 03.06.15 um 03:58 schrieb Jay K: >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> > > Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. > To me this is just another argument for a language specified value range of INTEGER. > It makes certain things easier. > > My suggestion was: > TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) > while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) > BITS 64 FOR INTEGER would only work on x64 systems. > What you want is already pretty much here. Just declare your desired type yourself: TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; It will always have this range and occupy 32 bits, regardless of the native word size. Pickles will always save and reread it with exactly this range, even between machines of different word size. The only thing that will differ from 32- to 64-bit machine is the size of intermediate results in expressions. If you are already _not_ being careful about avoiding overflows, there are probably cases where the results of silent overflows would differ with machine word size. Not sure what you would really want here. The unsigned arithmetic operations in Word are explicitly defined to silently do binary twos-complement wrap-around when overflows happen. The language does not define this for the builtin signed operators. In code where I care about overflows, I never assume this. It is probably machine-dependent. Or, you can import it: Int32.T. is already declared, in m3core, with this range. However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is almost never what you really want. The only place the BITS specification makes any difference is when you declare a record or object field of this type. In all other situations, BITS 32 FOR .. has _no effect_. The only purpose of BITS is so you can force the layout of a record. In that case, since the range needs exactly 32 bits anyway, even if you omit the BITS, a field of this type will still always occupy 32 bits. But, with the BITS specification, the compiler is not allowed to insert alignment padding, (if it were, you could not fully specify the layout) and if the field comes at a place that is not 32-bit aligned, you will either get a compile-time error or maybe a compiler will accept it and handle unaligned field access (it is not required to--ours does not), neither of which is probably what you want. Without BITS, the compiler will pad as needed. With BITS, you will have to insert explicit padding for alignment yourself as needed, which can be different for every field of this type of every record/object. And later adding/removing/changing any prior field can upset it and require you to rework the padding. Moreover, I am sure there are simple cases where the required padding differs between 32-bit and 64-bit machines. Having been bitten by this more than once, I now only put BITS..FOR as the anonymous type of a specific field, never in a named type to be used in multiple places. (Actually, BITS does the same thing when used as the type of an array element, but cases where this matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array fits in a single machine word. I do have a couple of such cases.) > The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value > range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for > any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). > > Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in > here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In > a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same > tends to be much slower). > > Regards, > Elmar > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Thu Jun 4 19:43:07 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Thu, 04 Jun 2015 19:43:07 +0200 Subject: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? In-Reply-To: <55707E3B.7020601@lcwb.coop> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> Message-ID: <55708E2B.4040607@elstel.org> Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem Am 04.06.15 um 18:35 schrieb Rodney M. Bates: > > > On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: >> Am 03.06.15 um 03:58 schrieb Jay K: >>> We cannot cross from 32bit host to 64bit target. >>> >>> >>> Ok to commit this? >>> >>> >> >> Compatibility between the 32bit and the 64bit version of the same >> program would be the minimum I expected from a pickle. >> To me this is just another argument for a language specified value >> range of INTEGER. >> It makes certain things easier. >> >> My suggestion was: >> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS >> 32 FOR INTEGER for 32 and 64bit targets) >> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I >> believe there is no reason to forbid it for INTEGER) >> BITS 64 FOR INTEGER would only work on x64 systems. >> > > What you want is already pretty much here. Just declare your desired > type yourself: > > TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; This should be good news. However my suggestion was to make Int32.T the default as this will be somewhat faster for the average application which is data intensive (just think of image processing) which will never be rewritten to use Int32.T instead of INT. All of our current and old programs being prepared to run on a 32bit as well as a 64bit target should compile and run without any problem. There is one single exception where the compiler would throw an error: when using INTEGER for address arithmetics in unsafe modules (error is: can only LOOPHOLE between types of the same size, or some way similar as I have already tested this). There we would need the so called targetint/offset. If I understand things right the following two things would need to get implemented to allow a default int size of 32 for existing applications (be it the default or just an additional compiler option as the size of WIDECHAR already is): * automatic range adaption for packed ints (i.e. those with BITS .. FOR) * allowing non 32bit alignment of INTs in memory (as is already the case for WIDE[CHAR]s) > > It will always have this range and occupy 32 bits, regardless of the > native word size. Pickles > will always save and reread it with exactly this range, even between > machines of different > word size. So the target size of the pickle is already determined by the respective INT subrange, right? Something that should to my believe be well documented as one could easily like to extend an enum with 255 members to more or equal than 256 an then wonder why the new an the old file formats become incompatible. To circumvent such a problem we would likely need to allow BITS 8*anything FOR INTEGER and take the bitcount specified in BITS for in order to determine the target size of an object instead of semi-automatically readjusting it all the time. > > The only thing that will differ from 32- to 64-bit machine is the size > of intermediate results in > expressions. If you are already _not_ being careful about avoiding > overflows, there are probably > cases where the results of silent overflows would differ with machine > word size. Not sure what > you would really want here. The unsigned arithmetic operations in > Word are explicitly defined > to silently do binary twos-complement wrap-around when overflows > happen. The language does not > define this for the builtin signed operators. In code where I care > about overflows, I never assume > this. It is probably machine-dependent. Yes, I believe it would be good like this. No real 32bit INT with 32bit arithmetics but just restricting the range for Int32.T when it gets written into memory. > > Or, you can import it: Int32.T. is already declared, in m3core, with > this range. > > However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is > almost never what you really want. > The only place the BITS specification makes any difference is when you > declare a record or object > field of this type. In all other situations, BITS 32 FOR .. has _no > effect_. The only purpose of > BITS is so you can force the layout of a record. In that case, since > the range needs exactly 32 > bits anyway, even if you omit the BITS, a field of this type will > still always occupy 32 bits. Nonetheless it is currently not possible to declare a variable of a 32bit type globally: TYPE Pixel = BITS 32 FOR INTEGER; unluckily does not work as we can have no global variable of type Pixel. This is not only un-orthogonal but also a very practical problem; Suggesting that we call procedures with a parameter of type pixel we will never be able to change that type f.i. to a packed record with members red, green, blue alpha if there are places in our main program where we need to declare VAR pix1, pix2 : INTEGER just because Pixel is a packed type being refused as global variable. > > But, with the BITS specification, the compiler is not allowed to > insert alignment padding, (if it > were, you could not fully specify the layout) and if the field comes > at a place that is not 32-bit > aligned, you will either get a compile-time error or maybe a compiler > will accept it and handle > unaligned field access (it is not required to--ours does not), neither > of which is probably what > you want. Without BITS, the compiler will pad as needed. With BITS, > you will have to insert explicit > padding for alignment yourself as needed, which can be different for > every field of this type of every > record/object. And later adding/removing/changing any prior field can > upset it and require you to > rework the padding. Moreover, I am sure there are simple cases where > the required padding differs > between 32-bit and 64-bit machines. On the long term I would personally even advocate an ALIGN directive: f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END; It would be very handy to have this for tapping the full power of the vectorization units of current processors. Manually re-aligning heap allocated objects in an unsafe module is not a very satisfying alternative (no local/global/stack allocation, memory waste and thus a certain performance loss if there are many wholes). However this would possibly also make changes to our garbage collector necessary as it currently only guarantees word alignment. Finally it would also be possible to write ALIGN BITSIZE(TARGETINT) .... > > Having been bitten by this more than once, I now only put BITS..FOR as > the anonymous type of a specific > field, never in a named type to be used in multiple places. A less bitchy and more orthogonal Modula-3 compiler supporting integer globals and locals of non word size and alignment would be a real pleasure, do you consent Rodney? > > (Actually, BITS does the same thing when used as the type of an array > element, but cases where this > matters are rare. If the bit count evenly divides the word size, > there won't be any alignment padding > needed, and BITS won't matter. Otherwise, the compiler is likely to > refuse, unless the entire array > fits in a single machine word. I do have a couple of such cases.) > >> The value range - bitsize correlation problem could be solved easily >> by automatically extending and reducing the value >> range of an integer type to what its binary representation allows as >> long as no explicit range has ever been specified for >> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR >> [-32768..+32767]). >> >> Finally it needs to be said that the argument that target size >> arithmetics is always the fastest which was often used in >> here can be very wrong. It does not hold for the 64bit systems of >> today because the CPU is no more the bottleneck. In >> a fact now the memory hierarchy has become the new bottleneck (which >> means that using more memory for the same >> tends to be much slower). >> >> Regards, >> Elmar >> > From mika at async.caltech.edu Thu Jun 4 19:46:29 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 04 Jun 2015 10:46:29 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55707E3B.7020601@lcwb.coop> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> Message-ID: <20150604174629.603F21A2065@async.async.caltech.edu> >(Actually, BITS does the same thing when used as the type of an array element, but cases where this >matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >fits in a single machine word. I do have a couple of such cases.) For a lot of hardware-related problems, the following type is quite useful: ARRAY [..] OF BITS 1 FOR BOOLEAN (and works with our compilers, too, as far as I know.) Mika From rodney_bates at lcwb.coop Thu Jun 4 19:51:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 12:51:43 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: <5570902F.3030401@lcwb.coop> On 06/04/2015 11:50 AM, Antony Hosking wrote: > >> On Jun 4, 2015, at 11:45 AM, Jay K > wrote: >> >> How strongly preferred and why? > > TInt preferred because LONGINT is still a terrible hack, and I hate to see it pollute the system more than it already has. > What?s more, the backends break on a number of operations for it. Danger, Will Robinson! This could introduce truly show-stopping bootstrap problems, using LONGINT in the compiler, and in places essential to its ability to function at all. One little bug and we could end up with neither a chicken nor an egg. > I now regret ever having introduced it, and would like to back it out. We would have been better to specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF INTEGER, and then hidden manipulation of those offsets behind a clean interface, much as Tint does. I think the one use-case we had for it to allow interfacing directly to C functions that take offset_t arguments could have been finessed differently. > >> good naturedly: >> >> >> 1) to cause me pain so that might give up? >> 2) to take more time so I don't muck with other stuff? >> 3) for compatibility with older releases? >> 4) for easier extension in future? >> 5) to cause me pain so I whine more about wanting operator overloading? > > Perhaps 3 & 4. > >> I started this once and it going to be a pain. >> LONGINT is probably much easier. >> Maybe I have more patience now. > > :-) Patience accrues with age? > >> Extension in the future could be addition of INTEGER128 to the language. :) >> And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128 >> and convert frontend to use it. Hypothetically. Perhaps before that happens >> we'll have operator overloading. :) > > Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. > >> >> >> - Jay >> >> >> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- --- >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Thu, 4 Jun 2015 11:32:48 -0400 >> CC: rodney.m.bates at acm.org ; m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> Again, TInt preferred. >> >> Sent from my iPhone >> >> On Jun 4, 2015, at 11:08 AM, Jay K > wrote: >> >> When I proposed TInt I had actually forgotten about LONGINT! >> LONGINT is easier to code to (until/unless we get operator overloading...) >> >> TInt is easier to extend, and is portable to before the current release. >> I think LONGINT is adequate. >> I did extent TInt to 72 bits I think. >> >> - Jay >> >> >> > Date: Thu, 4 Jun 2015 09:37:06 -0500 >> > From:rodney_bates at lcwb.coop >> > To:jay.krell at cornell.edu ;hosking at purdue.edu >> > CC:m3devel at elegosoft.com >> > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> > >> > >> > >> > On 06/03/2015 06:14 PM, Jay K wrote: >> > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. >> > > An array of chars? Probably has same problem. >> > > >> > > >> > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? >> > > >> > > - Jay >> > > >> > >> > I'm OK with doing it, but I thought you had been talking about using TInt. >> > Would that be better? It would be more general. But LONGINT would be >> > faster at compile time, and less work, since the arithmetic operators >> > would not need to be changed to TInt function calls. There might >> > be quite a lot of those I guess the compiler would be of about equal >> > help in finding missed places to be changed, either way. >> > >> > I just checked, and LONGINT is in the release compiler, contrary to >> > my sense of relative history, so there would be no bootstrap barrier. >> > >> > Either would allow a 32-bit hosted compiler (cross- or native-) to handle >> > types whose byte-count approached 2^31, instead of just 2^23. With a >> > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, >> > whereas LONGINT would only go to 2^55. Do we really care? >> > >> > > >> > > >> > > >> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----- >> > -- >> > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> > > From:hosking at purdue.edu >> > > Date: Wed, 3 Jun 2015 14:07:02 -0400 >> > > CC:m3devel at elegosoft.com >> > > To:jay.krell at cornell.edu >> > > >> > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. >> > > >> > > >> > -- >> > Rodney Bates >> >rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From adacore at marino.st Thu Jun 4 19:57:23 2015 From: adacore at marino.st (John Marino) Date: Thu, 04 Jun 2015 19:57:23 +0200 Subject: [M3devel] rpath woes Message-ID: <55709183.4060804@marino.st> I'm been finished with my port update for more than 1 day, except for one tiny problem. The rpath was wrong after I shifted the installation to /usr/local/cm3. I have tried just about every combination of changes to files at m3-sys/cminstall/src/config-no-install and I can't get the rpath to change. It's set at "/usr/local/lib", I don't know where it comes from. I want it to be "/usr/local/cm3/lib:/usr/local/lib". At gnuld.common, there are weird definitions of "-Wl,-rpath," but altering them doesn't help (not even "-W1,-rpath,//$ORIGIN"/../lib" So I am at a loss where rpath is getting set. If I knew that, I could change it. It seems to ignore all this SYSTEM_LD stuff so it must not be set. Any hints or do I have to keep randomly changing anything like looks like it could be a linker flag? John From jay.krell at cornell.edu Thu Jun 4 20:01:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 18:01:05 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, <55706292.9060608@lcwb.coop>, , <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu>, , Message-ID: > Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. The combination of features I want hasn't been materialized. Something like: 1) operator overloading -- C++ 2) templates with type deduction (i.e. C++ function templates, not just class templates or generic modules) -- C++ 3) a small easy to understand language -- not C++, maybe Modula-3 4) a small easy to understand compiler (m3front I don't understand) -- nowhere 5) fast compilation -- Modula-3 6) optional safey -- Modula-3 7) multiple inheritance, at least of "interfaces" -- C++ 8) optionally stack or inline allocation of "classes"/"objects" -- C++ 9) clear choice of build system -- Modula-3 perhaps 10) locals with destructors -- C++ 11) clear portable choice for remoting/RPC -- Modula-3 perhaps 12) needs backend work -- Modula-3 perhaps 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta 14) ?safety w/o garbage collection? -- rust?? 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. Not Python/Perl/Ruby/Erlang/sh/Tcl. 16) static compilation to native code, maybe -- C++ and Modula-3! Go? Net.Native? 17) non-virtual member functions -- not C or Modula-3 18) virtual member functions -- C++, Modula-3 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ was late here. 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 and C are decades late. 22) good string library 23) good regex library Basically, for now, I want static compilation to native code, fast compilation, optional safety. That combination is rare. Rust is unusual in safety w/o gc. Extended static lifetime analysis/verification... - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 12:50:02 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu On Jun 4, 2015, at 11:45 AM, Jay K wrote: How strongly preferred and why? TInt preferred because LONGINT is still a terrible hack, and I hate to see it pollute the system more than it already has. What?s more, the backends break on a number of operations for it. I now regret ever having introduced it, and would like to back it out. We would have been better to specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF INTEGER, and then hidden manipulation of those offsets behind a clean interface, much as Tint does. I think the one use-case we had for it to allow interfacing directly to C functions that take offset_t arguments could have been finessed differently. good naturedly: 1) to cause me pain so that might give up? 2) to take more time so I don't muck with other stuff? 3) for compatibility with older releases? 4) for easier extension in future? 5) to cause me pain so I whine more about wanting operator overloading? Perhaps 3 & 4. I started this once and it going to be a pain. LONGINT is probably much easier. Maybe I have more patience now. :-) Patience accrues with age? Extension in the future could be addition of INTEGER128 to the language. :)And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128and convert frontend to use it. Hypothetically. Perhaps before that happenswe'll have operator overloading. :) Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 11:32:48 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu Again, TInt preferred. Sent from my iPhone On Jun 4, 2015, at 11:08 AM, Jay K wrote: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 20:05:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 18:05:00 +0000 Subject: [M3devel] rpath woes In-Reply-To: <55709183.4060804@marino.st> References: <55709183.4060804@marino.st> Message-ID: My understanding: The modula-3 libraries, e.g. m3core,are found from the rpath I put in, like $ORIGIN:$ORIGIN/../lib Other libraries like libc.so are found by full path, that is already correct at build time? Do users want to possibly have a private /usr/local/lib/libc.so?I hadn't considered that. Or build is "staged" and the full paths aren't correct at build time?I think I hadn't considered that either. You are on the right track, looking in this area. Did you notice the "portable rpath" thingy that make-dist.py changes? - Jay > Date: Thu, 4 Jun 2015 19:57:23 +0200 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] rpath woes > > I'm been finished with my port update for more than 1 day, except for > one tiny problem. The rpath was wrong after I shifted the installation > to /usr/local/cm3. > > I have tried just about every combination of changes to files at > m3-sys/cminstall/src/config-no-install and I can't get the rpath to change. > > It's set at "/usr/local/lib", I don't know where it comes from. I want > it to be "/usr/local/cm3/lib:/usr/local/lib". > > At gnuld.common, there are weird definitions of "-Wl,-rpath," but > altering them doesn't help (not even "-W1,-rpath,//$ORIGIN"/../lib" > > So I am at a loss where rpath is getting set. If I knew that, I could > change it. It seems to ignore all this SYSTEM_LD stuff so it must not > be set. > > Any hints or do I have to keep randomly changing anything like looks > like it could be a linker flag? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Thu Jun 4 20:10:07 2015 From: adacore at marino.st (John Marino) Date: Thu, 04 Jun 2015 20:10:07 +0200 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st> Message-ID: <5570947F.2080600@marino.st> On 6/4/2015 20:05, Jay K wrote: > My understanding: > The modula-3 libraries, e.g. m3core, > are found from the rpath I put in, like > $ORIGIN:$ORIGIN/../lib > Other libraries like libc.so are found by full path, that is already > correct at build time? > Do users want to possibly have a private /usr/local/lib/libc.so? > I hadn't considered that. > Or build is "staged" and the full paths aren't correct at build time? > I think I hadn't considered that either. > You are on the right track, looking in this area. > Did you notice the "portable rpath" thingy that make-dist.py changes? I believe make-dist.py is hard-coding portable rpath which means it's hardcoding $ORIGIN I think, but it's not working. The origin flag is set, but $ORIGIN itself is not. I think this is a bug. We prefer to use hardcoded rpaths in ports, but I will settle for $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not NetBSD (not sure about OpenBSD, I'd guess it does though). The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't work either. John From hendrik at topoi.pooq.com Thu Jun 4 20:24:44 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 4 Jun 2015 14:24:44 -0400 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st> Message-ID: <20150604182444.GB8703@topoi.pooq.com> On Thu, Jun 04, 2015 at 06:05:00PM +0000, Jay K wrote: > Do users want to possibly have a private /usr/local/lib/libc.so?I > hadn't considered that. The musl users might. It's another C runtime system which may have some standards-compliance and efficiency advantages. see http://www.musl-libc.org/ -- hemdrik From lists at darko.org Thu Jun 4 20:32:17 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 4 Jun 2015 11:32:17 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: You really should try writing your own language, it's really not that hard. M3 will never give you satisfaction. You've already implemented a C back end and no doubt have an idea of how a compiler works. As for language features, I think we should be looking for features to remove. On Thu, Jun 4, 2015 at 11:01 AM, Jay K wrote: > > Operator overloading is anathema to the Modula-3 design principles. > Use C++ if that is what you want. > > > The combination of features I want hasn't been materialized. > Something like: > > 1) operator overloading -- C++ > 2) templates with type deduction (i.e. C++ function templates, not just > class templates or generic modules) -- C++ > 3) a small easy to understand language -- not C++, maybe Modula-3 > 4) a small easy to understand compiler (m3front I don't understand) -- > nowhere > 5) fast compilation -- Modula-3 > 6) optional safey -- Modula-3 > 7) multiple inheritance, at least of "interfaces" -- C++ > 8) optionally stack or inline allocation of "classes"/"objects" -- C++ > 9) clear choice of build system -- Modula-3 perhaps > 10) locals with destructors -- C++ > 11) clear portable choice for remoting/RPC -- Modula-3 perhaps > 12) needs backend work -- Modula-3 perhaps > 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta > 14) ?safety w/o garbage collection? -- rust?? > 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. > Not Python/Perl/Ruby/Erlang/sh/Tcl. > 16) static compilation to native code, maybe -- C++ and Modula-3! Go? > Net.Native? > 17) non-virtual member functions -- not C or Modula-3 > 18) virtual member functions -- C++, Modula-3 > 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) > 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ > was late here. > 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 > and C are decades late. > 22) good string library > 23) good regex library > > > Basically, for now, I want static compilation to native code, fast > compilation, optional safety. > That combination is rare. > > > Rust is unusual in safety w/o gc. Extended static lifetime > analysis/verification... > > > - Jay > > > > ------------------------------ > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > From: hosking at purdue.edu > Date: Thu, 4 Jun 2015 12:50:02 -0400 > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > To: jay.krell at cornell.edu > > > On Jun 4, 2015, at 11:45 AM, Jay K wrote: > > How strongly preferred and why? > > > TInt preferred because LONGINT is still a terrible hack, and I hate to see > it pollute the system more than it already has. What?s more, the backends > break on a number of operations for it. I now regret ever having > introduced it, and would like to back it out. We would have been better to > specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF > INTEGER, and then hidden manipulation of those offsets behind a clean > interface, much as Tint does. I think the one use-case we had for it to > allow interfacing directly to C functions that take offset_t arguments > could have been finessed differently. > > good naturedly: > > > 1) to cause me pain so that might give up? > 2) to take more time so I don't muck with other stuff? > 3) for compatibility with older releases? > 4) for easier extension in future? > 5) to cause me pain so I whine more about wanting operator overloading? > > > Perhaps 3 & 4. > > I started this once and it going to be a pain. > LONGINT is probably much easier. > Maybe I have more patience now. > > > :-) Patience accrues with age? > > Extension in the future could be addition of INTEGER128 to the language. :) > And the 128 bit targets will initially have 64bit LONGINT limits, until we > add INTEGER128 > and convert frontend to use it. Hypothetically. Perhaps before that happens > we'll have operator overloading. :) > > > Operator overloading is anathema to the Modula-3 design principles. Use > C++ if that is what you want. > > > > - Jay > > > ------------------------------ > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > From: hosking at purdue.edu > Date: Thu, 4 Jun 2015 11:32:48 -0400 > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > To: jay.krell at cornell.edu > > Again, TInt preferred. > > Sent from my iPhone > > On Jun 4, 2015, at 11:08 AM, Jay K wrote: > > When I proposed TInt I had actually forgotten about LONGINT! > LONGINT is easier to code to (until/unless we get operator overloading...) > > TInt is easier to extend, and is portable to before the current release. > I think LONGINT is adequate. > I did extent TInt to 72 bits I think. > > - Jay > > > > Date: Thu, 4 Jun 2015 09:37:06 -0500 > > From: rodney_bates at lcwb.coop > > To: jay.krell at cornell.edu; hosking at purdue.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > > > > > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > > sorry, I missed that it is literals...so I can convert a blueray movie > into a source file containing the data all in a text? Far fetched. > > > An array of chars? Probably has same problem. > > > > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and > field offsets? > > > > > > - Jay > > > > > > > I'm OK with doing it, but I thought you had been talking about using > TInt. > > Would that be better? It would be more general. But LONGINT would be > > faster at compile time, and less work, since the arithmetic operators > > would not need to be changed to TInt function calls. There might > > be quite a lot of those I guess the compiler would be of about equal > > help in finding missed places to be changed, either way. > > > > I just checked, and LONGINT is in the release compiler, contrary to > > my sense of relative history, so there would be no bootstrap barrier. > > > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > > types whose byte-count approached 2^31, instead of just 2^23. With a > > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > -- > > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > > > From: hosking at purdue.edu > > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > > CC: m3devel at elegosoft.com > > > To: jay.krell at cornell.edu > > > > > > It?s only TEXT /literals/ that are limited here. As in, what appears > in a source program. > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 5 00:24:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 17:24:26 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <20150604174629.603F21A2065@async.async.caltech.edu> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> <20150604174629.603F21A2065@async.async.caltech.edu> Message-ID: <5570D01A.5000909@lcwb.coop> On 06/04/2015 12:46 PM, mika at async.caltech.edu wrote: >> (Actually, BITS does the same thing when used as the type of an array element, but cases where this >> matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >> needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >> fits in a single machine word. I do have a couple of such cases.) > > For a lot of hardware-related problems, the following type is quite useful: > > ARRAY [..] OF BITS 1 FOR BOOLEAN > > (and works with our compilers, too, as far as I know.) > Yeah, I forgot about that group of cases. Without a BITS spec, the only sizes the compiler will choose are 64, 32, 16, or 8. If you want 4, 2, or 1, or something else that doesn't divide 32, you have to use BITS. But they will work fine. > Mika > -- Rodney Bates rodney.m.bates at acm.org From adacore at marino.st Fri Jun 5 00:28:36 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 00:28:36 +0200 Subject: [M3devel] rpath woes In-Reply-To: <5570947F.2080600@marino.st> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> Message-ID: <5570D114.40104@marino.st> On 6/4/2015 20:10, John Marino wrote: > On 6/4/2015 20:05, Jay K wrote: >> My understanding: >> The modula-3 libraries, e.g. m3core, >> are found from the rpath I put in, like > >> $ORIGIN:$ORIGIN/../lib >> Other libraries like libc.so are found by full path, that is already >> correct at build time? >> Do users want to possibly have a private /usr/local/lib/libc.so? >> I hadn't considered that. >> Or build is "staged" and the full paths aren't correct at build time? >> I think I hadn't considered that either. >> You are on the right track, looking in this area. >> Did you notice the "portable rpath" thingy that make-dist.py changes? > > I believe make-dist.py is hard-coding portable rpath which means it's > hardcoding $ORIGIN I think, but it's not working. > > The origin flag is set, but $ORIGIN itself is not. I think this is a bug. > > We prefer to use hardcoded rpaths in ports, but I will settle for > $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not > NetBSD (not sure about OpenBSD, I'd guess it does though). > > The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't > work either. Okay, I have this solved. The problem is FREEBSD.common is obsolete (origin linker flags were wrong) I basically copied LINUX.common contents to it and now everything works as expected. I've updated the port here: http://www.freshports.org/lang/modula3/ The patch for FREEBSD.common is here: https://svnweb.freebsd.org/ports/head/lang/modula3/files/ I think all references to FREEBSD4 could be removed (as I did in the patch). FreeBSD4 was EOL in Jan 2007. So I'm basically done until its time to change the port to use the C backend (and/or bootstrap it to DragonFly) John From jay.krell at cornell.edu Fri Jun 5 06:16:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 5 Jun 2015 04:16:24 +0000 Subject: [M3devel] rpath woes In-Reply-To: <5570D114.40104@marino.st> References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>,<5570D114.40104@marino.st> Message-ID: The FreeBSD4 users were surprisingly vocal surprisingly recently. So I put some work into it. I agree it is an old system. Linux.common and FreeBSD.common look very similar but I agree different and if it is working for you, good, I can ignore it? I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? Was this right for FreeBSD 5 or 6 or 7 or 8? Dragonfly should be fairly easy...though it is a bit more work than is ideal. The main thing is we still have lists of targets. Look in m3-sys/m3middle/src/Target.* Soon hopefully the jmpbuf there will go away. That will help. Areas to port are easy to find with grep/findstr: C:\dev2\cm3\m3-libs\libm3>findstr /m /s /i OPENBSD * src\os\POSIX\m3makefile src\os\POSIX\m3makefile-old src\uid\POSIX\getMID.c src\uid\POSIX\MachineIDPosixC.c tests\random\src\m3makefile C:\dev2\cm3\m3-libs\m3core>findstr /m /s /i OPENBSD * src\atomic\m3makefile src\C\Common\Cstring.i3 src\Csupport\Common\lround-readme.txt src\Csupport\Common\s_llroundl.c src\Csupport\Common\s_lround.c src\Csupport\Common\s_lroundl.c src\Csupport\m3makefile-old src\float\m3makefile src\float\m3makefile-old src\m3core.h src\runtime\POSIX\RTSignalC.c src\thread\POSIX\ThreadPosixC.c src\thread\PTHREAD\m3makefile src\thread\PTHREAD\ThreadOpenBSD.c src\thread\PTHREAD\ThreadPThreadC.c src\time\POSIX\m3makefile.old src\unix\Common\Ustat.i3 src\unix\m3makefile src\unix\m3makefile-old C:\dev2\cm3\m3-sys\m3middle>findstr /m /s /i OPENBSD * src\Target.i3 src\Target.i3-old src\Target.m3 src\Target.m3-old This looks scarier than it is. Many cases are just to add to a list of platforms you are close enough to. Some cases I hope to eliminate -- cooperative suspend will cut out a swath for example. Thanks, - Jay > Date: Fri, 5 Jun 2015 00:28:36 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > > On 6/4/2015 20:10, John Marino wrote: > > On 6/4/2015 20:05, Jay K wrote: > >> My understanding: > >> The modula-3 libraries, e.g. m3core, > >> are found from the rpath I put in, like > > > >> $ORIGIN:$ORIGIN/../lib > >> Other libraries like libc.so are found by full path, that is already > >> correct at build time? > >> Do users want to possibly have a private /usr/local/lib/libc.so? > >> I hadn't considered that. > >> Or build is "staged" and the full paths aren't correct at build time? > >> I think I hadn't considered that either. > >> You are on the right track, looking in this area. > >> Did you notice the "portable rpath" thingy that make-dist.py changes? > > > > I believe make-dist.py is hard-coding portable rpath which means it's > > hardcoding $ORIGIN I think, but it's not working. > > > > The origin flag is set, but $ORIGIN itself is not. I think this is a bug. > > > > We prefer to use hardcoded rpaths in ports, but I will settle for > > $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not > > NetBSD (not sure about OpenBSD, I'd guess it does though). > > > > The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't > > work either. > > Okay, I have this solved. > The problem is FREEBSD.common is obsolete (origin linker flags were > wrong) I basically copied LINUX.common contents to it and now > everything works as expected. > > I've updated the port here: > http://www.freshports.org/lang/modula3/ > > The patch for FREEBSD.common is here: > https://svnweb.freebsd.org/ports/head/lang/modula3/files/ > > I think all references to FREEBSD4 could be removed (as I did in the > patch). FreeBSD4 was EOL in Jan 2007. > > So I'm basically done until its time to change the port to use the C > backend (and/or bootstrap it to DragonFly) > > John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jun 5 07:50:33 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 07:50:33 +0200 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>, <5570D114.40104@marino.st> Message-ID: <557138A9.3000605@marino.st> On 6/5/2015 06:16, Jay K wrote: > The FreeBSD4 users were surprisingly vocal surprisingly recently. > So I put some work into it. > I agree it is an old system. It's completely unsupported (for 7 years now) and they don't need the latest M3 right? Old release still work fine, that should be enough justification to change this on trunk. > Linux.common and FreeBSD.common look very similar but I agree different > and if it is working for you, good, I can ignore it? > I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? > Was this right for FreeBSD 5 or 6 or 7 or 8? yes. I am using the latest binutils (thus latest ld) with M3. The base ld is ancient and hand maintained (a GPLv3 issue) so I don't know, Actually, the linux way is opposite: It explicitly defines SYSTEM_LD rather than use flags for SYSTEM_CC. So rather than flags, we are talking about the approach. Really nobody with a supported FreeBSD release needs to build from scratch because it exists in ports. If anybody absolutely wants to, just say "emulates what the port is doing". This is actually easier said than done because in addition to using the latest binutils, there are a number of inline substitutions that are done, see post-patch target here, starting line 78: https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup line 85 is where newer gcc and newer as is set for m3. John From adacore at marino.st Fri Jun 5 08:10:08 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 08:10:08 +0200 Subject: [M3devel] install issues (stripping, permissions) Message-ID: <55713D40.6080305@marino.st> This is old news but I never reported it. See the install target on the port's makefile (starting line 100): https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup I have to due numerous changes to the installed files to make it conform to ports standards. The issues involved stripping and permissions. For ports, executables are installed stripped unless explicitly wanted. Usually gnu makefiles support this with "install" and "install-strip" targets. This can be done after the fact, as I have done, with the ${STRIP_CMD} command, which does nothing when stripping is not desired. The other issue is that there were numerous files in pkg/ that were executable and shouldn't have been. I used find/chmod to remove exec perms for all except "cm3" file. I did this 1.5 years ago, I'm now wondering how right it was. The current list: pkg/m3core/src/C/Common/Csetjmp.i3 pkg/m3back/src/M3C.i3 pkg/m3staloneback/AMD64_FREEBSD/m3back pkg/cm3/AMD64_FREEBSD/cm3 pkg/libdump/AMD64_FREEBSD/libdump pkg/cmpfp/AMD64_FREEBSD/cmpfp pkg/formsview/AMD64_FREEBSD/formsview pkg/vorun/AMD64_FREEBSD/vorun pkg/pkl-fonts/AMD64_FREEBSD/PklFonts pkg/hack/AMD64_FREEBSD/dummy pkg/test/AMD64_FREEBSD/test Should there be executables in pkg/ ? Is the chmod command then wrong? I had to strips these as well to pass QA tests. John From wagner at elegosoft.com Fri Jun 5 09:26:18 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 5 Jun 2015 09:26:18 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55713D40.6080305@marino.st> References: <55713D40.6080305@marino.st> Message-ID: <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> On Fri, 05 Jun 2015 08:10:08 +0200 John Marino wrote: > The other issue is that there were numerous files in pkg/ that were > executable and shouldn't have been. I used find/chmod to remove exec > perms for all except "cm3" file. I did this 1.5 years ago, I'm now > wondering how right it was. The current list: > > pkg/m3core/src/C/Common/Csetjmp.i3 > pkg/m3back/src/M3C.i3 > pkg/m3staloneback/AMD64_FREEBSD/m3back > pkg/cm3/AMD64_FREEBSD/cm3 > pkg/libdump/AMD64_FREEBSD/libdump > pkg/cmpfp/AMD64_FREEBSD/cmpfp > pkg/formsview/AMD64_FREEBSD/formsview > pkg/vorun/AMD64_FREEBSD/vorun > pkg/pkl-fonts/AMD64_FREEBSD/PklFonts > pkg/hack/AMD64_FREEBSD/dummy > pkg/test/AMD64_FREEBSD/test > > Should there be executables in pkg/ ? Is the chmod command then wrong? > I had to strips these as well to pass QA tests. Except for the i3-files these are all programs and should be executable. The builder ships programs to the package target directory, and Programs with a capital P to the bin directory. As I said, M3 packages don't fit well into OS packages. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From estellnb at elstel.org Fri Jun 5 10:21:31 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:21:31 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <7427033D-A18B-4F81-A823-229F91313F16@purdue.edu> References: <55705A41.7070503@elstel.org> <7427033D-A18B-4F81-A823-229F91313F16@purdue.edu> Message-ID: <55715C0B.8090207@elstel.org> Antony, you do not seem to understand what I have proposed. Please have a look at my latest mail about Re: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? Am 04.06.15 um 19:14 schrieb Antony Hosking: > >> On Jun 4, 2015, at 10:01 AM, Elmar Stellnberger > > wrote: >> >> Am 03.06.15 um 03:58 schrieb Jay K: >>> We cannot cross from 32bit host to 64bit target. >>> >>> >>> Ok to commit this? >>> >>> >> >> Compatibility between the 32bit and the 64bit version of the same >> program would be the minimum I expected from a pickle. >> To me this is just another argument for a language specified value >> range of INTEGER. >> It makes certain things easier. >> >> My suggestion was: >> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS >> 32 FOR INTEGER for 32 and 64bit targets) >> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I >> believe there is no reason to forbid it for INTEGER) >> BITS 64 FOR INTEGER would only work on x64 systems. > > You are still misunderstanding the purpose of BITS FOR. It is used to > express packing requirements for use of those types *inside* > structured values (such as RECORD or ARRAY) to enforce a particular > bit-packing and to avoid alignment contraints. What it means is that > bit-wise operations must be used to load/store the value in memory in > the case that aligned operations do not suffice. > >> >> The value range - bitsize correlation problem could be solved easily >> by automatically extending and reducing the value >> range of an integer type to what its binary representation allows as >> long as no explicit range has ever been specified for >> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR >> [-32768..+32767]). > > A value of type [-32768..+32767] *never* occupies more than 16 bits in > memory! > >> Finally it needs to be said that the argument that target size >> arithmetics is always the fastest which was often used in >> here can be very wrong. It does not hold for the 64bit systems of >> today because the CPU is no more the bottleneck. In >> a fact now the memory hierarchy has become the new bottleneck (which >> means that using more memory for the same >> tends to be much slower). > > Operating on 16-bit values using natural word-size operations (32 bit > or 64 bit) is never more expensive. And the memory is only ever > 16-bit. So your performance argument does not stand. > > I hope this clarifies your understanding of the Modula-3 type system. > >> >> Regards, >> Elmar >> >>> Can't do this. Non-open arrays must have static constant bounds, >>> which the above does not. Open arrays have extra dope and are accessed >>> through two extra pointers, which we really wouldn't want for text >>> literals. >>> >>> TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse >>> that anything >>> else in UNSAFE code. The interface is UNSAFE, which means safe code >>> can't IMPORT >>> it, and thus can't see the declaration. Not very likely that >>> anybody other than >>> the runtime, compiler-generated code, and debugger (written in C >>> anyway) would >>> want to mess with it. >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Fri Jun 5 10:24:32 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:24:32 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: <55715CC0.401@elstel.org> Am 04.06.15 um 20:32 schrieb Darko Volaric: > You really should try writing your own language, it's really not that > hard. Darko, you are dreaming. Writing a small demo compiler may be one weeks job. However writing something that implements all that your hart desires will be very hard to do if at all doable for a single person. > M3 will never give you satisfaction. You've already implemented a C > back end and no doubt have an idea of how a compiler works. > > As for language features, I think we should be looking for features to > remove. > Jay, with a little more time I would have been at your side in a first place! A couple of years ago I had some own plans for a M3/Pascal like language with similar or even more venturous properties. 1) operator overloading -- C++ 2) templates with type deduction (i.e. C++ function templates, not just class templates or generic modules) -- C++ 3) a small easy to understand language -- not C++, maybe Modula-3 4) a small easy to understand compiler (m3front I don't understand) -- nowhere 5) fast compilation -- Modula-3 6) optional safey -- Modula-3 7) multiple inheritance, at least of "interfaces" -- C++ 8) optionally stack or inline allocation of "classes"/"objects" -- C++ 9) clear choice of build system -- Modula-3 perhaps 10) locals with destructors -- C++ 11) clear portable choice for remoting/RPC -- Modula-3 perhaps 12) needs backend work -- Modula-3 perhaps 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta 14) ?safety w/o garbage collection? -- rust?? 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. Not Python/Perl/Ruby/Erlang/sh/Tcl. 16) static compilation to native code, maybe -- C++ and Modula-3! Go? Net.Native? 17) non-virtual member functions -- not C or Modula-3 18) virtual member functions -- C++, Modula-3 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ was late here. 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 and C are decades late. 22) good string library 23) good regex library Nonetheless I wanna see what can be done with Modula-3 in its current state. - and I believe it already offers some interesting properties compared to C like f.i. range checking and better type safety which could make a difference for security relevant projects like f.i. a web browser. The only thing that would be necessary for such a project was a *** well tested and ready-to-use shipped *** GUI toolkit ... From estellnb at elstel.org Fri Jun 5 10:33:54 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:33:54 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55715CC0.401@elstel.org> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> <55715CC0.401@elstel.org> Message-ID: <55715EF2.7020500@elstel.org> > 17) non-virtual member functions -- not C or Modula-3 That already works for Modula-3. Simply put the method name with the same signature another time under "methods" instead of overrides (However you would have to do that in each subclass - and there is likely no runtime optimization for it; i.e. it may effectively just declare another virtual method with the same name.). From estellnb at elstel.org Fri Jun 5 10:29:24 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:29:24 +0200 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <556BD24D.7010902@lcwb.coop> References: <20150601002944.GA10751@topoi.pooq.com> <556BD24D.7010902@lcwb.coop> Message-ID: <55715DE4.3000304@elstel.org> Am 01.06.15 um 05:32 schrieb Rodney M. Bates: > There is a QT binding in cm3/m3-ui/qt. I have no experience > with it. > Dear Hendrik, I am also very interested in a good GUI environment for Modula-3. It would be very nice if you could share your experience with the current Qt bindings if you should resolve to give them a test. Once I had started to port the program at http://www.elstel.org/coan/ to Modula-3. That time there were no Qt-bindings and I had started to program my own bindings for M3. As far as I can remember they use virtual methods instead of signals or slots. I am very likely to continue my work on it at least if the current Qt toolkit should not be as good as desired. By the end of this summer I would suppose CoAn to be available under an OSSy license and being ported to Modula-3. Elmar > On 05/31/2015 07:29 PM, Hendrik Boom wrote: >> What user interface libraries are available for Modula 3? >> >> I know there's Trestle. >> >> But is there something like GTK or QT or somethng I can use like cairo >> to draw really pretty text? Anything that supports lots of Unicode? >> I don't mind havein to do my own character placement based on font >> metrics of some sort; not now, bit later, I may have some really >> weird layout constraints. >> >> In case you're wodering about the application: >> >> I'm doing preliminary planning for something like a text editor with >> several windows, one with the raw text, and another with a continuous >> (as continuous as I have cpu cycles for) display of the results of >> rather >> complex calculations on that text (such as proof checking or >> described graphics). >> >> Modula 3 isn't the only candidate for a programming language, just the >> leading candidate for historical and bootstrapping reasons. The others >> at the moment are OCAML and some kind of statically typed Scheme. >> >> And of course, I may decide that theo whole projecct is infeasible. >> >> -- hendrik >> >> > From adacore at marino.st Fri Jun 5 11:14:50 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 11:14:50 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> Message-ID: <5571688A.4060404@marino.st> On 6/5/2015 09:26, Olaf Wagner wrote: > On Fri, 05 Jun 2015 08:10:08 +0200 > John Marino wrote: > >> The other issue is that there were numerous files in pkg/ that were >> executable and shouldn't have been. I used find/chmod to remove exec >> perms for all except "cm3" file. I did this 1.5 years ago, I'm now >> wondering how right it was. The current list: >> >> pkg/m3core/src/C/Common/Csetjmp.i3 >> pkg/m3back/src/M3C.i3 >> pkg/m3staloneback/AMD64_FREEBSD/m3back >> pkg/cm3/AMD64_FREEBSD/cm3 >> pkg/libdump/AMD64_FREEBSD/libdump >> pkg/cmpfp/AMD64_FREEBSD/cmpfp >> pkg/formsview/AMD64_FREEBSD/formsview >> pkg/vorun/AMD64_FREEBSD/vorun >> pkg/pkl-fonts/AMD64_FREEBSD/PklFonts >> pkg/hack/AMD64_FREEBSD/dummy >> pkg/test/AMD64_FREEBSD/test >> >> Should there be executables in pkg/ ? Is the chmod command then wrong? >> I had to strips these as well to pass QA tests. > > Except for the i3-files these are all programs and should be executable. Okay, when I fixed this internally, I discovered something: m3back, libdump, cmpfp, formsview, vorun, dummy, test all link to m3 libraries but the rpath of $ORIGIN:$ORIGIN/../lib isn't enough. All of these programs need an additional rpath to $ORIGIN/../../../lib for the linker to find the libs. These all need to be fixed in repository I suspect. John From lists at darko.org Fri Jun 5 13:07:20 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 5 Jun 2015 04:07:20 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55715CC0.401@elstel.org> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> <55715CC0.401@elstel.org> Message-ID: I guess it depends on your level of skills, but source to source compilers aren't that challenging. The more important point is that exercise might focus Jay's mind on the the complexity and desirability of those features. It's all fine talking about them in the abstract, it's completely different when you get down to brass tacks. On Fri, Jun 5, 2015 at 1:24 AM, Elmar Stellnberger wrote: > Am 04.06.15 um 20:32 schrieb Darko Volaric: > >> You really should try writing your own language, it's really not that >> hard. >> > Darko, you are dreaming. Writing a small demo compiler may be one weeks > job. > However writing something that implements all that your hart desires will > be > very hard to do if at all doable for a single person. > > M3 will never give you satisfaction. You've already implemented a C back >> end and no doubt have an idea of how a compiler works. >> >> As for language features, I think we should be looking for features to >> remove. >> >> Jay, with a little more time I would have been at your side in a first > place! > A couple of years ago I had some own plans for a M3/Pascal like language > with similar or even more venturous properties. > > 1) operator overloading -- C++ > 2) templates with type deduction (i.e. C++ function templates, not just > class templates or generic modules) -- C++ > 3) a small easy to understand language -- not C++, maybe Modula-3 > 4) a small easy to understand compiler (m3front I don't understand) -- > nowhere > 5) fast compilation -- Modula-3 > 6) optional safey -- Modula-3 > 7) multiple inheritance, at least of "interfaces" -- C++ > 8) optionally stack or inline allocation of "classes"/"objects" -- C++ > 9) clear choice of build system -- Modula-3 perhaps > 10) locals with destructors -- C++ > 11) clear portable choice for remoting/RPC -- Modula-3 perhaps > 12) needs backend work -- Modula-3 perhaps > 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta > 14) ?safety w/o garbage collection? -- rust?? > 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. > Not Python/Perl/Ruby/Erlang/sh/Tcl. > 16) static compilation to native code, maybe -- C++ and Modula-3! Go? > Net.Native? > 17) non-virtual member functions -- not C or Modula-3 > 18) virtual member functions -- C++, Modula-3 > 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) > 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ > was late here. > 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 > and C are decades late. > 22) good string library > 23) good regex library > > Nonetheless I wanna see what can be done with Modula-3 in its current > state. - > and I believe it already offers some interesting properties compared to C > like > f.i. range checking and better type safety which could make a difference > for > security relevant projects like f.i. a web browser. The only thing that > would be > necessary for such a project was a *** well tested and ready-to-use > shipped *** > GUI toolkit ... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Jun 5 17:42:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 05 Jun 2015 08:42:17 -0700 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>, <5570D114.40104@marino.st> Message-ID: <20150605154217.AABC81A2062@async.async.caltech.edu> Jay K writes: >--_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > The FreeBSD4 users were surprisingly vocal surprisingly recently. > So I put some work into it. > I agree it is an old system. I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, but the 4.x config worked well. I've since upgraded to 10.0-RELEASE. And don't use it much anymore because posix threads (still) don't work right on FreeBSD. I spent a considerable effort to get them working right on Debian, which they now do, but didn't have the time to figure it out for a bunch of other OSes as well. It's really pretty nice to have them working correctly. They aren't perfect (a bit too much locking over garbage collection issues) but they don't ever seem to crash. On problems with lots of parallelism you do get some parallel speedup, even. Mika From jay.krell at cornell.edu Fri Jun 5 20:51:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 5 Jun 2015 18:51:13 +0000 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <5571688A.4060404@marino.st> References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st> Message-ID: This is kind of on purpose as Olaf said and kind of broken as you point out. We often build_standalone() to workaround the origin/rpath matte. We kind of fail here, in terms of library paths. I don't think the original system fully accounted for runpath. I think at the time, LD_LIBRARY_PATH might have been considered an acceptable solution. Elsewhere, with libtool, make install relinks to reset paths. We don't have that implemented. It is somewhat a good solution. - Jay > Date: Fri, 5 Jun 2015 11:14:50 +0200 > From: adacore at marino.st > To: wagner at elegosoft.com > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] install issues (stripping, permissions) > > On 6/5/2015 09:26, Olaf Wagner wrote: > > On Fri, 05 Jun 2015 08:10:08 +0200 > > John Marino wrote: > > > >> The other issue is that there were numerous files in pkg/ that were > >> executable and shouldn't have been. I used find/chmod to remove exec > >> perms for all except "cm3" file. I did this 1.5 years ago, I'm now > >> wondering how right it was. The current list: > >> > >> pkg/m3core/src/C/Common/Csetjmp.i3 > >> pkg/m3back/src/M3C.i3 > >> pkg/m3staloneback/AMD64_FREEBSD/m3back > >> pkg/cm3/AMD64_FREEBSD/cm3 > >> pkg/libdump/AMD64_FREEBSD/libdump > >> pkg/cmpfp/AMD64_FREEBSD/cmpfp > >> pkg/formsview/AMD64_FREEBSD/formsview > >> pkg/vorun/AMD64_FREEBSD/vorun > >> pkg/pkl-fonts/AMD64_FREEBSD/PklFonts > >> pkg/hack/AMD64_FREEBSD/dummy > >> pkg/test/AMD64_FREEBSD/test > >> > >> Should there be executables in pkg/ ? Is the chmod command then wrong? > >> I had to strips these as well to pass QA tests. > > > > Except for the i3-files these are all programs and should be executable. > > Okay, when I fixed this internally, I discovered something: > m3back, libdump, cmpfp, formsview, vorun, dummy, test all link to m3 > libraries but the rpath of $ORIGIN:$ORIGIN/../lib isn't enough. All of > these programs need an additional rpath to $ORIGIN/../../../lib for the > linker to find the libs. These all need to be fixed in repository I > suspect. > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jun 5 22:53:14 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 22:53:14 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st> Message-ID: <55720C3A.5070200@marino.st> On 6/5/2015 20:51, Jay K wrote: > This is kind of on purpose as Olaf said and kind of broken as you point out. > > We often build_standalone() to workaround the origin/rpath matte. > > We kind of fail here, in terms of library paths. well, to be blunt, it -is- broken and thus is a fail. > I don't think the original system fully accounted for runpath. > I think at the time, LD_LIBRARY_PATH might have been considered an > acceptable solution. If I wanted to go down a path like this, I'd use ldconfig, but my personal feeling if is ldconfig is required, the software is doing something wrong. > Elsewhere, with libtool, make install relinks to reset paths. We don't > have that implemented. > It is somewhat a good solution. I think it has to be a tweak at FreeBSD.common -- rather than use $ORIGIN/../lib for rpath, it needs a conditional statement to use $ORIGIN/../../../lib instead. I just don't know what the condition is. I'm also trying to figure out why rpath=$ORIGIN is needed. are there ever libraries in the same directory as cm3? ohn From jay.krell at cornell.edu Sat Jun 6 00:59:55 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 15:59:55 -0700 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55720C3A.5070200@marino.st> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> Message-ID: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> We could use origin, origin/../lib, origin/../../lib etc? But eventually a security problem to reach all around? Plain origin is in case the files are all together in one directory like on windows. Isn't it desirable to be buildable w/ system cc/ld and latest? Thank you for this work. - Jay On Jun 5, 2015, at 1:53 PM, John Marino wrote: > On 6/5/2015 20:51, Jay K wrote: >> This is kind of on purpose as Olaf said and kind of broken as you point out. >> >> We often build_standalone() to workaround the origin/rpath matte. >> >> We kind of fail here, in terms of library paths. > > well, to be blunt, it -is- broken and thus is a fail. > >> I don't think the original system fully accounted for runpath. >> I think at the time, LD_LIBRARY_PATH might have been considered an >> acceptable solution. > > If I wanted to go down a path like this, I'd use ldconfig, but my > personal feeling if is ldconfig is required, the software is doing > something wrong. > > >> Elsewhere, with libtool, make install relinks to reset paths. We don't >> have that implemented. >> It is somewhat a good solution. > > I think it has to be a tweak at FreeBSD.common -- rather than use > $ORIGIN/../lib for rpath, it needs a conditional statement to use > $ORIGIN/../../../lib instead. I just don't know what the condition is. > > I'm also trying to figure out why rpath=$ORIGIN is needed. are there > ever libraries in the same directory as cm3? > > ohn From jay.krell at cornell.edu Sat Jun 6 01:01:48 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 16:01:48 -0700 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> Message-ID: <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> Also "dist" means "distribution" means run on other machines, try not to be too specific to build machine. That is why not full paths. - Jay On Jun 5, 2015, at 3:59 PM, Jay wrote: > We could use origin, origin/../lib, origin/../../lib etc? But eventually a security problem to reach all around? Plain origin is in case the files are all together in one directory like on windows. > > Isn't it desirable to be buildable w/ system cc/ld and latest? > > Thank you for this work. > > - Jay > > On Jun 5, 2015, at 1:53 PM, John Marino wrote: > >> On 6/5/2015 20:51, Jay K wrote: >>> This is kind of on purpose as Olaf said and kind of broken as you point out. >>> >>> We often build_standalone() to workaround the origin/rpath matte. >>> >>> We kind of fail here, in terms of library paths. >> >> well, to be blunt, it -is- broken and thus is a fail. >> >>> I don't think the original system fully accounted for runpath. >>> I think at the time, LD_LIBRARY_PATH might have been considered an >>> acceptable solution. >> >> If I wanted to go down a path like this, I'd use ldconfig, but my >> personal feeling if is ldconfig is required, the software is doing >> something wrong. >> >> >>> Elsewhere, with libtool, make install relinks to reset paths. We don't >>> have that implemented. >>> It is somewhat a good solution. >> >> I think it has to be a tweak at FreeBSD.common -- rather than use >> $ORIGIN/../lib for rpath, it needs a conditional statement to use >> $ORIGIN/../../../lib instead. I just don't know what the condition is. >> >> I'm also trying to figure out why rpath=$ORIGIN is needed. are there >> ever libraries in the same directory as cm3? >> >> ohn From jay.krell at cornell.edu Sat Jun 6 02:20:44 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 17:20:44 -0700 Subject: [M3devel] rpath woes In-Reply-To: <20150605154217.AABC81A2062@async.async.caltech.edu> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> <5570D114.40104@marino.st> <20150605154217.AABC81A2062@async.async.caltech.edu> Message-ID: <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> Posix threads from C? From Modula-3? - Jay On Jun 5, 2015, at 8:42 AM, wrote: > Jay K writes: >> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> The FreeBSD4 users were surprisingly vocal surprisingly recently. >> So I put some work into it. >> I agree it is an old system. > > I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, > but the 4.x config worked well. > > I've since upgraded to 10.0-RELEASE. And don't use it much anymore > because posix threads (still) don't work right on FreeBSD. I spent a > considerable effort to get them working right on Debian, which they > now do, but didn't have the time to figure it out for a bunch of > other OSes as well. It's really pretty nice to have them working > correctly. They aren't perfect (a bit too much locking over garbage > collection issues) but they don't ever seem to crash. On problems > with lots of parallelism you do get some parallel speedup, even. > > Mika From mika at async.caltech.edu Sat Jun 6 02:32:10 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 05 Jun 2015 17:32:10 -0700 Subject: [M3devel] rpath woes In-Reply-To: <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> <5570D114.40104@marino.st> <20150605154217.AABC81A2062@async.async.caltech.edu> <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> Message-ID: <20150606003210.50F2E1A2065@async.async.caltech.edu> I think I've mentioned this on this mailing list about a hundred times, but... the pthreads implementation of Modula-3 threads doesn't work on most OSes. As far as I know the only OS where it is truly reliably working is AMD64_LINUX. I certainly haven't tested all targets. User-level threads work everywhere. Except for AMD64_LINUX I would not suggest putting anything important to use Modula-3 threads built on pthreads. They work... until they don't. Anyone wishing to work on this and fix it, please don't change the AMD64_LINUX code at all unless you know exactly, precisely what you are doing. It is very very difficult to debug the subtle bugs that get introduced. As far as I know there's no problem with C pthreads on FreeBSD. Mika Jay writes: >Posix threads from C? From Modula-3? > > - Jay > >On Jun 5, 2015, at 8:42 AM, wrote: > >> Jay K writes: >>> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >>> Content-Type: text/plain; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> The FreeBSD4 users were surprisingly vocal surprisingly recently. >>> So I put some work into it. >>> I agree it is an old system. >> >> I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, >> but the 4.x config worked well. >> >> I've since upgraded to 10.0-RELEASE. And don't use it much anymore >> because posix threads (still) don't work right on FreeBSD. I spent a >> considerable effort to get them working right on Debian, which they >> now do, but didn't have the time to figure it out for a bunch of >> other OSes as well. It's really pretty nice to have them working >> correctly. They aren't perfect (a bit too much locking over garbage >> collection issues) but they don't ever seem to crash. On problems >> with lots of parallelism you do get some parallel speedup, even. >> >> Mika From adacore at marino.st Sat Jun 6 07:53:40 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 07:53:40 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> Message-ID: <55728AE4.4030301@marino.st> On 6/6/2015 01:01, Jay wrote: > Also "dist" means "distribution" means run on other machines, try not to be too specific to build machine. That is why not full paths. > > - Jay > > On Jun 5, 2015, at 3:59 PM, Jay wrote: >>> I think it has to be a tweak at FreeBSD.common -- rather than use >>> $ORIGIN/../lib for rpath, it needs a conditional statement to use >>> $ORIGIN/../../../lib instead. I just don't know what the condition is. >>> >>> I'm also trying to figure out why rpath=$ORIGIN is needed. are there >>> ever libraries in the same directory as cm3? >>> >>> ohn I think you misunderstood. I know why "$ORIGIN/../lib" is there. I don't know why "$ORIGIN" is there. There are no libraries in /usr/local/cm3/bin, so why is this directory specified? John From adacore at marino.st Sat Jun 6 07:57:50 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 07:57:50 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> Message-ID: <55728BDE.8030702@marino.st> On 6/6/2015 00:59, Jay wrote: > We could use origin, origin/../lib, origin/../../lib etc? But > eventually a security problem to reach all around? Plain origin is in > case the files are all together in one directory like on windows. I'm not really a fan of just adding every possible rpath. In this case, it's either $ORIGIN/../lib or $ORIGIN/../../../lib, but together is never correct (as the other mail indicates, $ORIGIN by itself also seems to be always wrong) > > Isn't it desirable to be buildable w/ system cc/ld and latest? On DragonFly - yes On FreeBSD - no. FreeBSD is stuck with either ancient gcc (4.2.1), ancient binutils (2.17) or clang. In every case, it needs to use newer GCC and binutils from ports to build M3. (and every newer GCC) John From jay.krell at cornell.edu Sat Jun 6 10:47:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 6 Jun 2015 08:47:25 +0000 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55728BDE.8030702@marino.st> References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st>, , <55720C3A.5070200@marino.st>, <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com>, <55728BDE.8030702@marino.st> Message-ID: $ORIGIN is "in case" we restructure Unix install to look like Windows install -- moving lib on top of bin. I realize you'd like just the minimum required, not hypothetical values for the future. Regarding gcc/ld -- isn't it desirable to have either/both work? Are there command lines that work with both and provide the same meaning? Does our stuff not work with older gcc/ld? Do ports in general avoid system gcc/ld? Does clang work? I intend to try. - Jay > Date: Sat, 6 Jun 2015 07:57:50 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] install issues (stripping, permissions) > > On 6/6/2015 00:59, Jay wrote: > > We could use origin, origin/../lib, origin/../../lib etc? But > > eventually a security problem to reach all around? Plain origin is in > > case the files are all together in one directory like on windows. > > I'm not really a fan of just adding every possible rpath. In this case, > it's either $ORIGIN/../lib or $ORIGIN/../../../lib, but together is > never correct (as the other mail indicates, $ORIGIN by itself also seems > to be always wrong) > > > > > Isn't it desirable to be buildable w/ system cc/ld and latest? > > On DragonFly - yes > On FreeBSD - no. > > FreeBSD is stuck with either ancient gcc (4.2.1), ancient binutils > (2.17) or clang. In every case, it needs to use newer GCC and binutils > from ports to build M3. (and every newer GCC) > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jun 6 10:48:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 6 Jun 2015 08:48:41 +0000 Subject: [M3devel] rpath woes In-Reply-To: <20150606003210.50F2E1A2065@async.async.caltech.edu> References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>,<5570D114.40104@marino.st> , <20150605154217.AABC81A2062@async.async.caltech.edu>, <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com>, <20150606003210.50F2E1A2065@async.async.caltech.edu> Message-ID: I thought these problems were all fixed by now. :( Maybe cooperative suspend will help. I know that seems like a non-sequiter, but I know that SuspendThread and GetThreadContext on Windows don't do what you'd expect and as a result our x86-on-amd64 code isn't correct. - Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > Date: Fri, 5 Jun 2015 17:32:10 -0700 > From: mika at async.caltech.edu > > I think I've mentioned this on this mailing list about a hundred times, but... > > the pthreads implementation of Modula-3 threads doesn't work on most OSes. > > As far as I know the only OS where it is truly reliably working is AMD64_LINUX. > > I certainly haven't tested all targets. > > User-level threads work everywhere. > > Except for AMD64_LINUX I would not suggest putting anything important to > use Modula-3 threads built on pthreads. They work... until they don't. > > Anyone wishing to work on this and fix it, please don't change the > AMD64_LINUX code at all unless you know exactly, precisely what you > are doing. It is very very difficult to debug the subtle bugs that > get introduced. > > As far as I know there's no problem with C pthreads on FreeBSD. > > Mika > > Jay writes: > >Posix threads from C? From Modula-3? > > > > - Jay > > > >On Jun 5, 2015, at 8:42 AM, wrote: > > > >> Jay K writes: > >>> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ > >>> Content-Type: text/plain; charset="iso-8859-1" > >>> Content-Transfer-Encoding: quoted-printable > >>> > >>> The FreeBSD4 users were surprisingly vocal surprisingly recently. > >>> So I put some work into it. > >>> I agree it is an old system. > >> > >> I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, > >> but the 4.x config worked well. > >> > >> I've since upgraded to 10.0-RELEASE. And don't use it much anymore > >> because posix threads (still) don't work right on FreeBSD. I spent a > >> considerable effort to get them working right on Debian, which they > >> now do, but didn't have the time to figure it out for a bunch of > >> other OSes as well. It's really pretty nice to have them working > >> correctly. They aren't perfect (a bit too much locking over garbage > >> collection issues) but they don't ever seem to crash. On problems > >> with lots of parallelism you do get some parallel speedup, even. > >> > >> Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sat Jun 6 14:02:29 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 14:02:29 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st>, , <55720C3A.5070200@marino.st>, <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com>, <55728BDE.8030702@marino.st> Message-ID: <5572E155.2040408@marino.st> On 6/6/2015 10:47, Jay K wrote: > $ORIGIN is "in case" we restructure Unix install to look like Windows > install -- moving lib on top of bin. > I realize you'd like just the minimum required, not hypothetical values > for the future. Since it's patched anyway, I'll just remove $ORIGIN then. Obviously I'm going to vote against a lowest-common-denomination approach of dumping everything into one directory. A better way is have the build system support either. > Regarding gcc/ld -- isn't it desirable to have either/both work? Given scarce developer resources (and small user population), I recommend that you target supporting ports rather than try to support multiple configurations. Unless you are really committed to supported platforms that have been EOL for years (basically FreeBSD 8.x and lower) then I don't see a big ROI on supporting more than what ports requires. To specifically answer, I'd say "no". FreeBSD 9, 10, 11 all have different base compiler configurations. It's not worth trying to support the quirks of each one. > Are there command lines that work with both and provide the same meaning? I'm not sure, but since ports works on FreeBSD 9 and later, I am not that motivated to do the extensive testing necessary to answer that. > Does our stuff not work with older gcc/ld? > Do ports in general avoid system gcc/ld? More and more. The freebsd base assembler is ancient, so often a new gas is required which brings in the new ld and forces that ports GCC be used (if not base clang) > Does clang work? > I intend to try. I expect it to work on the C-backend. John From mika at async.caltech.edu Sat Jun 6 18:50:23 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 06 Jun 2015 09:50:23 -0700 Subject: [M3devel] the need for cooperative suspend In-Reply-To: References: <20150502155713.29DCC1A2062@async.async.caltech.edu> Message-ID: <20150606165023.78AEC1A2066@async.async.caltech.edu> Can the thread tester survive with the most paranoid options on OS X? I don't believe it until I've seen it (based on what we went through for Linux, I would say I'm just being prudent). Run it with: "-iters 100000 -n 20 -tests ALL" I remember the thread tester died on FreeBSD last time I tried, and there have been no relevant checkins since that I know of. Here's what happens when I run it on my FreeBSD. CM3 is not up to date, in case someone fixed something I'm not aware of, so don't draw too many conclusions until someone has tried it on a fresh install: ..........laziest thread is 1433608799/9/1 (tests: read 10/6/6 nxread 26/6/4 tryexcept 67/53/38 fork 1433608799/2/2 forktoomuch 1433608799/9/3 alloc 1433608799/37/1 creat 10/7/7 lock 1433608799/68/38) *** *** runtime error: *** Segmentation violation - possible attempt to dereference NIL *** pc = 0x4387e8 = Move + 0x6a in ../src/runtime/common/RTCollector.m3 *** Abort (79)pluto:/tmp> Anyone who's interested in trying their own favorite Modula-3 installation, this is easy to do: cm3/m3-libs/m3core/tests/thread/AMD64_FREEBSD/threadtest -iters 100000 -n 20 -tests ALL Ignore the warnings about "thread being starved or deadlocked". Those should really be an option and not on by default: as far as I know the threading library makes no claim of fairness, so I don't think one should expect it by default. Mika Antony Hosking writes: >OS X is stable as well as Linux, as far as I know. >Mika, what misbehaviors are you seeing? > >Sent from my iPad > >> On May 2, 2015, at 11:57 AM, mika at async.caltech.edu wrote: >>=20 >> Jay,=20 >>=20 >> Can you explain precisely what you mean by "suspend"? >>=20 >> A thread is suspended by ... a timer expiration? Another thread at higher= > priority needing resources? >>=20 >> I'm thinking about a tight loop "for(;;) ;"... how do you stop a thread fr= >om running that (it could have >> been coded in assembly...)? >>=20 >> Jay K writes: >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_ >>> Content-Type: text/plain; charset=3D"iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>>=20 >>> We really need to move away from preemptive suspend. >>>=20 >>> 1) It is a large fraction of the target-dependent code in m3core. >>> =3D20 >>>=20 >>> 2) It doesn't really work in "wow64"=3D2C the x86-on-amd64 Windows system= >. >>> The SuspendThread + GetThreadContext does not actually reliably give you >>> current and coherent context. >>> I'm not sure about x86-on-ia64. >>> =3D20 >>>=20 >>> The context can be old. >>> The context can be mid-update. Which might be ok=3D2C depending on what t= >he =3D >>> updates are. >>> LIke if Eip and Esp are being updated=3D2C we don't care about Eip anyway= >. >>> =3D20 >>>=20 >>> You can find out if either is the case=3D2C and let the thread run longer= >=3D2C >>> but being in a long running syscall I believe is reported as possibly >>> invalid even though it is valid. >>> =3D20 >>> =3D20 >>> NT/native is ok. >>> =3D20 >>>=20 >>> We should just use cooperative suspend and not worry about it. >>> =3D20 >>> =3D20 >>> Thanks=3D2C >>> - Jay >>>=20 >>>=20 >>> =3D20 >>> =3D >>>=20 >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_ >>> Content-Type: text/html; charset=3D"iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>>=20 >>> >>> >>> >>>
We really need to move awa= >y from=3D >>> preemptive suspend.

1) It is a large fraction of the target-depend= >e=3D >>> nt code in m3core.
 =3D3B

2) It doesn't really work in "wow= >64"=3D >>> =3D2C the x86-on-amd64 Windows system.
The SuspendThread + GetThreadCo= >ntex=3D >>> t does not actually reliably give you
current and coherent context.>I=3D >>> 'm not sure about x86-on-ia64.
 =3D3B

 =3D3BThe context= > can b=3D >>> e old.
 =3D3BThe context can be mid-update. Which might be ok=3D2C= > depen=3D >>> ding on what the updates are.
 =3D3BLIke if Eip and Esp are being u= >pda=3D >>> ted=3D2C we don't care about Eip anyway.
 =3D3B

You can fin= >d out =3D >>> if either is the case=3D2C and let the thread run longer=3D2C
but bein= >g in a=3D >>> long running syscall I believe is reported as possibly
invalid even th= >o=3D >>> ugh it is valid.
 =3D3B
 =3D3B
NT/native is ok.
 = >=3D3B>>>
We should just use cooperative suspend and not worry about it.
&n= >bs=3D >>> p=3D3B
 =3D3B
Thanks=3D2C
 =3D3B- Jay


 =3D= >3B
=3D >>>
>>> =3D >>>=20 >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_-- From adacore at marino.st Sat Jun 6 23:32:52 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 23:32:52 +0200 Subject: [M3devel] Follow up - FreeBSD port Message-ID: <55736704.6030900@marino.st> Based on recent discussions, I altered the FreeBSD port as follows: 1) I removed the rpath=$ORIGIN flag 2) I limited the de-execution to the executable .i3 files 3) I moved the executable pkg programs to bin, then added softlinks to where the were originally installed. The last one enables those programs to work again, even from the pkg/*/AMD64_FREEBSD/ locations. I could not figure out a way to dynamically change the rpath so I just decided to use $ORIGIN/../lib for everything an make it work. It has a side benefit of eliminating the large, duplicate cm3 at pkg/cm3/AMD64_FREEBSD/cm3 (replaced by symlink). I think the port is in pretty good shape right now. Most of the programs work, but cm3ide core dumps a lot. It also seems to be querying a database that doesn't exist (pgsql installed but port doesn't do anything with it). Anything that tries to use TCP seems to fail. John http://www.freshports.org/commit.php?category=lang&port=modula3&files=yes&message_id=201506062130.t56LUF55083786 at svn.freebsd.org From mika at async.caltech.edu Sun Jun 7 08:05:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 06 Jun 2015 23:05:17 -0700 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <55736704.6030900@marino.st> References: <55736704.6030900@marino.st> Message-ID: <20150607060517.0FF6E1A2067@async.async.caltech.edu> John Marino writes: ... >Anything that tries to use TCP seems to fail. ... Might be a wild goose chase.. but I seem to remember having issues if the reverse and forward DNS of your machine don't match up, or if there is no reverse DNS... or something like that. Otherwise, TCP should work just fine. Mika From hendrik at topoi.pooq.com Sun Jun 7 14:47:12 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 7 Jun 2015 08:47:12 -0400 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <20150607060517.0FF6E1A2067@async.async.caltech.edu> References: <55736704.6030900@marino.st> <20150607060517.0FF6E1A2067@async.async.caltech.edu> Message-ID: <20150607124712.GA1045@topoi.pooq.com> On Sat, Jun 06, 2015 at 11:05:17PM -0700, mika at async.caltech.edu wrote: > John Marino writes: > ... > >Anything that tries to use TCP seems to fail. > ... > > Might be a wild goose chase.. but I seem to remember having issues if > the reverse and forward DNS of your machine don't match up, or if there > is no reverse DNS... or something like that. That's not TCP doing that as far as I know. Even DNS is built on top of TCP and IP, and TCP knows nothing of it. However, a lot of email servers perform such a check to block some forms of malfeasance. Possibly some other systems built on top of TCP do that as well. > > Otherwise, TCP should work just fine. > > Mika From mika at async.caltech.edu Sun Jun 7 18:03:50 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 07 Jun 2015 09:03:50 -0700 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <20150607124712.GA1045@topoi.pooq.com> References: <55736704.6030900@marino.st> <20150607060517.0FF6E1A2067@async.async.caltech.edu> <20150607124712.GA1045@topoi.pooq.com> Message-ID: <20150607160350.D5C1F1A2066@async.async.caltech.edu> Hendrik Boom writes: >On Sat, Jun 06, 2015 at 11:05:17PM -0700, mika at async.caltech.edu wrote: >> John Marino writes: >> ... >> >Anything that tries to use TCP seems to fail. >> ... >> >> Might be a wild goose chase.. but I seem to remember having issues if >> the reverse and forward DNS of your machine don't match up, or if there >> is no reverse DNS... or something like that. > >That's not TCP doing that as far as I know. Even DNS is built >on top of TCP and IP, and TCP knows nothing of it. > >However, a lot of email servers perform such a check to block some >forms of malfeasance. Possibly some other systems built on top of >TCP do that as well. > >> >> Otherwise, TCP should work just fine. >> >> Mika Well, yeah, it's obviously not TCP itself---don't think anyone claimed as much. It's something about the Modula-3 code that wraps TCP socket code making assumptions about the configuration of the host it's running on (that that host is configured in a "reasonable way" for a workstation in an industrial research lab, perhaps...) I remember running into it a few times and don't know if I (or anyone) ever fixed it in the distribution. Mika From rodney_bates at lcwb.coop Sun Jun 7 19:32:59 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 07 Jun 2015 12:32:59 -0500 Subject: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? In-Reply-To: <55708E2B.4040607@elstel.org> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> <55708E2B.4040607@elstel.org> Message-ID: <5574804B.5030304@lcwb.coop> On 06/04/2015 12:43 PM, Elmar Stellnberger wrote: > Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > Am 04.06.15 um 18:35 schrieb Rodney M. Bates: >> >> >> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: >>> Am 03.06.15 um 03:58 schrieb Jay K: >>>> We cannot cross from 32bit host to 64bit target. >>>> >>>> >>>> Ok to commit this? >>>> >>>> >>> >>> Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. >>> To me this is just another argument for a language specified value range of INTEGER. >>> It makes certain things easier. >>> >>> My suggestion was: >>> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) >>> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) >>> BITS 64 FOR INTEGER would only work on x64 systems. >>> >> >> What you want is already pretty much here. Just declare your desired type yourself: >> >> TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; > This should be good news. However my suggestion was to make Int32.T the default > as this will be somewhat faster for the average application which is data intensive (just > think of image processing) which will never be rewritten to use Int32.T instead of INT. > All of our current and old programs being prepared to run on a 32bit as well as a 64bit > target should compile and run without any problem. There is one single exception > where the compiler would throw an error: when using INTEGER for address arithmetics > in unsafe modules (error is: can only LOOPHOLE between types of the same size, or > some way similar as I have already tested this). There we would need the so called > targetint/offset. This would be a very biased thing to do. The only people who would benefit would be those who know for certain that they want *every* integer variable to have 32-bit range, on both 32-bit and 64-bit machines. Without debating the wisdom or likelihood of this, a quite easy string search/replace on "INTEGER" would accomplish the same thing. But everybody else would have to vet their integers one by one, to see which they wanted always 32 and which they wanted native size. It would break lots of code for the unlucky (majority, I would guess), while providing only speed improvements for the favored. I have worked on lots of code in the M3 distribution itself that would break. > If I understand things right the following two things would need to get implemented > to allow a default int size of 32 for existing applications (be it the default or just an > additional compiler option as the size of WIDECHAR already is): > * automatic range adaption for packed ints (i.e. those with BITS .. FOR) No, your are still trying to misuse BITS to control the value range. Subranges do this just fine. Use the mechanism the language already provides. BITS is for manual control of record/object/array layout only, not value range. > * allowing non 32bit alignment of INTs in memory (as is already the case for WIDE[CHAR]s) INTEGERs (and subranges thereof) are currently no different from WIDECHARS, in regard to alignment. They get the same treatment as anything else. For BITS types inside a record/object/array, the language lets the compiler choose what it will accept, but I am reasonably sure all existing M3 compiler variations, for all targets, will lay things out as you ask (including misaligned) and will correctly access them, as long as no item crosses a native word boundary. Outside records/objects/arrays, the compiler just aligns as needed. >> >> It will always have this range and occupy 32 bits, regardless of the native word size. Pickles >> will always save and reread it with exactly this range, even between machines of different >> word size. > So the target size of the pickle is already determined by the respective INT subrange, right? Defining what pickles do/should do is fraught with pitfalls. Here's what we now have: For (almost) all types, pickles preserve the properties of the M3 type. The bounds of a subrange type are static constant numbers, and are evaluated at compile time on the compiling machine. So [0..16_3FFFFFFF] is the same type on either size of machine, but [0..LAST(INTEGER) DIV 2] is not. On a 32-bit machine, [0..LAST(INTEGER) DIV 2] evaluates to, and thus means [0..16_3FFFFFFF], but on a 64-bit machine, it means [0..16_3FFFFFFFFFFFFFFF], a different type. So, for example, you could exchange pickled values between 32-bit and 64-bit machines if the field/element of the containing record/object/array type were written as [0..16_3FFFFFFF] on both machines, or if it was written as [0..LAST(INTEGER) DIV 2] on the 32-bit machine, and [0..16_3FFFFFFF] on the 64, but not if both had coded the type as [0..LAST(INTEGER) DIV 2]. INTEGER, CARDINAL, LONGINT, and LONGCARD are exceptions. Each of these is treated as the same M3 type, regardless of native word size and upper bound. So pickles will copy a field/element of any these types with a size change, when written and read on different sized machines. The size change does the more-or-less obvious sign-extend one way and raises an exception if the value won't fit, when going the other way. The floating types are not handled by pickles across machines with different floating formats. If the format is the same, (and one pickles knows exists), they will work, otherwise, they will raise an exception. It would be nice to implement this, but apparently, nobody has had a sufficient need. I am no numerical analyst, but I think defining good usable rules for this would call for the services of someone who is. > Something that should to my believe be well documented as one could easily like to extend > an enum with 255 members to more or equal than 256 an then wonder why the new an the > old file formats become incompatible. To circumvent such a problem we would likely need > to allow BITS 8*anything FOR INTEGER and take the bitcount specified in BITS for in order to > determine the target size of an object instead of semi-automatically readjusting it all the > time. This one really gets into a tar pit. Pickles encode the type of a value as a "fingerprint", which is a 64-bit has of the M3 type structure. A copy of this goes into the pickle for every type it contains. Meanwhile, the compiler puts copies of the fingerprints of all types in the program into static global readonly data. Pickle read uses this to look or a type defined in the reading program with the same signature, which means it must be exactly the same type, for the read to succeed. Allowing similar but unequal types would very difficult to define even what differences would be tolerated. This get very complicated very fast. Remember, there is no guarantee that the reading program is the same program as the writing program, or has much similarity to it at all. Pickles requires only that the reading program have a defined reference type that is structurally identical to that of the object had in the writing program. Allowing to write a field of type {Red, Green} in a pickle and read it into a program that has {Red, Green, Blue} would get very messy. Remember that pickles does not handle scalar values, just heap objects. So we would have to be defining some recursive rule about record types whose corresponding field types were each sufficiently "similar", etc, etc. And to implement, the pickle format would have to be completely reworked to contain full type structural descriptions, not just signatures. >> >> The only thing that will differ from 32- to 64-bit machine is the size of intermediate results in >> expressions. If you are already _not_ being careful about avoiding overflows, there are probably >> cases where the results of silent overflows would differ with machine word size. Not sure what >> you would really want here. The unsigned arithmetic operations in Word are explicitly defined >> to silently do binary twos-complement wrap-around when overflows happen. The language does not >> define this for the builtin signed operators. In code where I care about overflows, I never assume >> this. It is probably machine-dependent. > Yes, I believe it would be good like this. No real 32bit INT with 32bit arithmetics but just restricting > the range for Int32.T when it gets written into memory. >> >> Or, you can import it: Int32.T. is already declared, in m3core, with this range. >> >> However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is almost never what you really want. >> The only place the BITS specification makes any difference is when you declare a record or object >> field of this type. In all other situations, BITS 32 FOR .. has _no effect_. The only purpose of >> BITS is so you can force the layout of a record. In that case, since the range needs exactly 32 >> bits anyway, even if you omit the BITS, a field of this type will still always occupy 32 bits. > Nonetheless it is currently not possible to declare a variable of a 32bit type globally: > > TYPE Pixel = BITS 32 FOR INTEGER; > > unluckily does not work as we can have no global variable of type Pixel. No, you have misunderstood. Any type BITS n OF T can be used anywhere T can be used, including global variables, local variables, parameters, etc. It's just that, except for a field/element, the addition of BITS... will have no effect, in other words, it won't change that number of bits or alignment the compiler actually allocates, from what the compiler would do if the type were given as just T. It will not make the declaration illegal. My recommendation not to use BITS in a named TYPE declaration was only because: 1) Outside of a record/object/array, it will make no difference anyway, and 2) Inside of a record/object/array, the effect it has can and often will be different every time, so you have to think about each such field/element and separately judge whether the BITS specification is right for that particular case. And that depends on what comes before it in the record/object/array. Also, there is full assignability between BITS n FOR T and T. So you don't have to put explicit conversions in. The one case there is not such assignability is between BITS n FOR T and BITS m FOR T, with n#m. So copying between two record fields of the same base type but different explicit BITS specifications of different sizes would require explicit conversion. > This is not only un-orthogonal but also a very practical problem; Suggesting that we call > procedures with a parameter of type pixel we will never be able to change that type f.i. to > a packed record with members red, green, blue alpha if there are places in our main > program where we need to declare VAR pix1, pix2 : INTEGER just because Pixel is a packed > type being refused as global variable. > >> >> But, with the BITS specification, the compiler is not allowed to insert alignment padding, (if it >> were, you could not fully specify the layout) and if the field comes at a place that is not 32-bit >> aligned, you will either get a compile-time error or maybe a compiler will accept it and handle >> unaligned field access (it is not required to--ours does not), neither of which is probably what >> you want. Without BITS, the compiler will pad as needed. With BITS, you will have to insert explicit >> padding for alignment yourself as needed, which can be different for every field of this type of every >> record/object. And later adding/removing/changing any prior field can upset it and require you to >> rework the padding. Moreover, I am sure there are simple cases where the required padding differs >> between 32-bit and 64-bit machines. > On the long term I would personally even advocate an ALIGN directive: > f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END; > > It would be very handy to have this for tapping the full power of the vectorization units > of current processors. Manually re-aligning heap allocated objects in an unsafe module > is not a very satisfying alternative (no local/global/stack allocation, memory waste and > thus a certain performance loss if there are many wholes). However this would possibly > also make changes to our garbage collector necessary as it currently only guarantees > word alignment. Finally it would also be possible to write ALIGN BITSIZE(TARGETINT) .... > > >> >> Having been bitten by this more than once, I now only put BITS..FOR as the anonymous type of a specific >> field, never in a named type to be used in multiple places. > A less bitchy and more orthogonal Modula-3 compiler supporting integer globals and > locals of non word size and alignment would be a real pleasure, do you consent Rodney? >> >> (Actually, BITS does the same thing when used as the type of an array element, but cases where this >> matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >> needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >> fits in a single machine word. I do have a couple of such cases.) >> >>> The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value >>> range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for >>> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). >>> >>> Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in >>> here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In >>> a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same >>> tends to be much slower). >>> >>> Regards, >>> Elmar >>> >> > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 9 03:44:07 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 8 Jun 2015 18:44:07 -0700 Subject: [M3devel] m3cg right shift signed? Message-ID: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> (Not at propercomputer but sending anyway.) Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? - Jay From hendrik at topoi.pooq.com Tue Jun 9 14:38:29 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 9 Jun 2015 08:38:29 -0400 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> Message-ID: <20150609123829.GA7874@topoi.pooq.com> On Mon, Jun 08, 2015 at 06:44:07PM -0700, Jay wrote: > (Not at propercomputer but sending anyway.) > > Is right shift of a signed type meant to be supported and well > defined in the IR? I thought not, and assert so in the C backend. But > it occurs now, from m3front/cg/set_range or such. Maybe change that > to not? Surely unsigned types are adequate here? Signed and unsigned right shifts are different on most hardware that supports them. Signed shifts do sign extension, so that the shifts of negative numbers remain negative. Unsigned shifts do zero extension. There are (once were?) machines where shifts are (were?) faster than division by powers of two. Not sure how this relates to whatever in Modula 3 might or might not require them, though. -- hendrik From jay.krell at cornell.edu Tue Jun 9 17:42:59 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 15:42:59 +0000 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <20150609123829.GA7874@topoi.pooq.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com>, <20150609123829.GA7874@topoi.pooq.com> Message-ID: Division is among the slowest instructions that operates on integers even on modern hardware. It is/was frequently completely absent, though maybe those days are past. 6502 and 65816 had no hardware multiply or divide, nor maybe signed shift. - Jay > Date: Tue, 9 Jun 2015 08:38:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cg right shift signed? > > On Mon, Jun 08, 2015 at 06:44:07PM -0700, Jay wrote: > > (Not at propercomputer but sending anyway.) > > > > Is right shift of a signed type meant to be supported and well > > defined in the IR? I thought not, and assert so in the C backend. But > > it occurs now, from m3front/cg/set_range or such. Maybe change that > > to not? Surely unsigned types are adequate here? > > Signed and unsigned right shifts are different on most hardware that > supports them. Signed shifts do sign extension, so that the shifts of > negative numbers remain negative. Unsigned shifts do zero extension. > > There are (once were?) machines where shifts are (were?) faster > than division by powers of two. > > Not sure how this relates to whatever in Modula 3 might or might not > require them, though. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 9 18:05:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 09 Jun 2015 11:05:51 -0500 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> Message-ID: <55770EDF.4030606@lcwb.coop> I'm a little unclear what you are asking. I see several places in M3CG_Ops.i3 which call for sign-extension, e.g., load, widen, extract* and any way of pushing on the IR stack. While signed right shift is clearly a good primitive to sign-extend, all these IR ops leave leave this to the code generator, which is good, because the available machine opcodes will be target dependent. As for set_range, this too leaves it to the code generator when the word whose bits are to be set is 32 or 64 bits. For smaller, CG lowers this itself, using more primitive bit-twiddling operators. But it looks like the shifts it generates must be zero-filling, to work right. As for the shift IR operators themselves, M3CG_Ops.i3 defines them as the same as the shifts in Word, none of which is sign-filling. On 06/08/2015 08:44 PM, Jay wrote: > (Not at propercomputer but sending anyway.) > > Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? > > - Jay > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 9 20:30:17 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 18:30:17 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? Message-ID: Is there any interest in maintaining 1) gcc/m3cg/parse.c support for gcc 4.3/4.5/4.6? It is slightly messy. 1b) gcc 4.2? This is where hypothetical ARM_DARWIN support is, last I checked, years ago. 2) gcc-4.3/4.5/4.6 in-tree? Or remove them to keep checkouts smaller? There was complaint as to our source tree size, and these directories do add a lot of size for little gain. They were historically useful to transition versions. I don't think it was wrong to have the temporary growth. All targets except for ARM_DARWIN default to gcc 4.7. Do people often/ever go back and compare/debug? Or want to retain that ability, with the current ease? Or ok to get the files from history, for that rare-to-never event? If we do maintain the gcc backend, I would likely import as gcc-5.1 etc., and recreate the problem, but it can be a "rolling" problem and fix. But I'm not keen on maintaining the gcc backend anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 20:32:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 18:32:03 +0000 Subject: [M3devel] ARM_DARWIN? Message-ID: ARM_DARWIN has never fully worked.I got to the point of c_compile/assemble/link cm3 on the target and then running cm3 failed.I don't remember why. I no longer have a system/toolset setup to test that. Does anyone have an iPhone or iPad (or simulator?) setup with ssh access and native C compiler, assembler, and linker?If so I'd like access to resume/finish that work. And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider deleting gcc-apple. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Jun 9 20:58:20 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 9 Jun 2015 11:58:20 -0700 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: Message-ID: I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that do the trick? On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: > ARM_DARWIN has never fully worked. > I got to the point of c_compile/assemble/link cm3 on the target and then > running cm3 failed. > I don't remember why. > > > I no longer have a system/toolset setup to test that. > > > Does anyone have an iPhone or iPad (or simulator?) setup with ssh access > and native C compiler, assembler, and linker? > If so I'd like access to resume/finish that work. > > > And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider > deleting gcc-apple. > > > Thank you, > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 21:29:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 19:29:41 +0000 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: , Message-ID: Yes that hardware will likely do -- if it can be setup for ssh and w/ native cc/as/ld.But I'd really rather someone else do the setup work. - Jay Date: Tue, 9 Jun 2015 11:58:20 -0700 Subject: Re: [M3devel] ARM_DARWIN? From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that do the trick? On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: ARM_DARWIN has never fully worked.I got to the point of c_compile/assemble/link cm3 on the target and then running cm3 failed.I don't remember why. I no longer have a system/toolset setup to test that. Does anyone have an iPhone or iPad (or simulator?) setup with ssh access and native C compiler, assembler, and linker?If so I'd like access to resume/finish that work. And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider deleting gcc-apple. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Jun 9 21:51:20 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 9 Jun 2015 12:51:20 -0700 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: Message-ID: Well, according to this < http://guides.macrumors.com/SSH_into_your_iPod_touch_(Windows) > it's jailbreak and then install OpenSSH, then access over wifi. I can do that easily enough. The bigger issue for remove development would be resetting the device. It's battery powered so it requires a physical access to reset. On Tue, Jun 9, 2015 at 12:29 PM, Jay K wrote: > Yes that hardware will *likely *do -- if it can be setup for ssh and w/ > native cc/as/ld. > But I'd really rather someone else do the setup work. > > - Jay > > > > ------------------------------ > Date: Tue, 9 Jun 2015 11:58:20 -0700 > Subject: Re: [M3devel] ARM_DARWIN? > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > > > I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that > do the trick? > > On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: > > ARM_DARWIN has never fully worked. > I got to the point of c_compile/assemble/link cm3 on the target and then > running cm3 failed. > I don't remember why. > > > I no longer have a system/toolset setup to test that. > > > Does anyone have an iPhone or iPad (or simulator?) setup with ssh access > and native C compiler, assembler, and linker? > If so I'd like access to resume/finish that work. > > > And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider > deleting gcc-apple. > > > Thank you, > - Jay > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Tue Jun 9 22:02:45 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 09 Jun 2015 22:02:45 +0200 Subject: [M3devel] cm3: what are *.mc files Message-ID: <55774665.8060303@elstel.org> What are *.mc - files? They appear in TARGET - directories; most of them are just called _m3main.mc but some of them have other names. I ask because I am writing a program which should recognize and clear object files. It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. Usually files of a specific type start with a 32bit magic; however the mc files all have different starting sequences. Is there still a straight forward way to recognize an .mc file just by its binary content? From jay.krell at cornell.edu Tue Jun 9 22:12:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 20:12:51 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <55774665.8060303@elstel.org> References: <55774665.8060303@elstel.org> Message-ID: I believe before the m3cc backend existed, these were actually C code. (i.e. C backend used to be the one and only) Then they morphed into the intermediate representation fed to cm3cgwhich is what they are now. See here: https://github.com/modula3/cm3/tree/master/m3-sys/m3middle/src/M3CG_BinWr.m3 and m3-sys/m3cc/gcc/gcc/m3cg/parse.c m3cgcat can dump the binary form into text. _m3main.mc is the intermediate file that holds main.There is a configuration option to use it, vs. _m3main.c.I switched to _m3main.c. This is perhaps more portable if using C++ anywhere andwant to get the constructors/destructors run (though all modern implementationsdon't care really). Try cm3 -keep to see them all. i.e. normally they get deleted. - Jay > Date: Tue, 9 Jun 2015 22:02:45 +0200 > From: estellnb at elstel.org > To: m3devel at elegosoft.com > Subject: [M3devel] cm3: what are *.mc files > > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear > object files. > It does not seem to be sufficient to check for uppercase directories > which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by > its binary content? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 22:14:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 20:14:03 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: References: <55774665.8060303@elstel.org>, Message-ID: ps: foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io again, see cm3 -keep, err better yet, cm3 -keep -verbose You can see it running cm3cg and as and rm. - Jay From: jay.krell at cornell.edu To: estellnb at elstel.org; m3devel at elegosoft.com Date: Tue, 9 Jun 2015 20:12:51 +0000 Subject: Re: [M3devel] cm3: what are *.mc files I believe before the m3cc backend existed, these were actually C code. (i.e. C backend used to be the one and only) Then they morphed into the intermediate representation fed to cm3cgwhich is what they are now. See here: https://github.com/modula3/cm3/tree/master/m3-sys/m3middle/src/M3CG_BinWr.m3 and m3-sys/m3cc/gcc/gcc/m3cg/parse.c m3cgcat can dump the binary form into text. _m3main.mc is the intermediate file that holds main.There is a configuration option to use it, vs. _m3main.c.I switched to _m3main.c. This is perhaps more portable if using C++ anywhere andwant to get the constructors/destructors run (though all modern implementationsdon't care really). Try cm3 -keep to see them all. i.e. normally they get deleted. - Jay > Date: Tue, 9 Jun 2015 22:02:45 +0200 > From: estellnb at elstel.org > To: m3devel at elegosoft.com > Subject: [M3devel] cm3: what are *.mc files > > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear > object files. > It does not seem to be sufficient to check for uppercase directories > which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by > its binary content? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 10 02:21:23 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 09 Jun 2015 19:21:23 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <55774665.8060303@elstel.org> References: <55774665.8060303@elstel.org> Message-ID: <55778303.3010009@lcwb.coop> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear object files. > It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by its binary content? > They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, or 16_FD 16_10 16_01, produced by a very recent head compiler. Ignore the 4th byte. > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 10 11:12:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:12:06 +0000 Subject: [M3devel] AMD64_DARWIN config Message-ID: I'd like to bring this back to what it was: diff --git a/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN b/m3-sys/cminstall/src/config-no-install/AMD64_DARWINindex c30dd35..8fa0a4a 100644--- a/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN+++ b/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN@@ -4,9 +4,13 @@ %readonly SYSTEM_CC = "/opt/local/bin/gcc-mp-4.7" %readonly SYSTEM_LIBTOOL = "/opt/local/bin/libtool"-readonly SYSTEM_CC = "/usr/bin/gcc"-readonly SYSTEM_LIBTOOL = "/usr/bin/libtool"-readonly SYSTEM_ASM = "/usr/bin/as"++% Darwin.common does a good job of determining these, esp. the assembler.+% Older /usr/bin/as default to x86 and fail for amd64.+%readonly SYSTEM_CC = "/usr/bin/gcc"+%readonly SYSTEM_LIBTOOL = "/usr/bin/libtool"+%readonly SYSTEM_ASM = "/usr/bin/as"+%readonly SYSTEM_ASM = "/usr/bin/as -arch x86_64" readonly TARGET = "AMD64_DARWIN" readonly GNU_PLATFORM = "i686-darwin8" % "cpu-os" string for GNU Does that really not work for people? Why?Are the cc/libtool on $PATH not acceptable? /usr/bin/as does not work on older systems. At least not w/o -arch. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 11:45:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:45:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? Message-ID: set_range appears to fail to set any bitsif requested to set one bit in a set that doesn't fit in machine word. Test case is here:https://github.com/modula3/cm3/tree/master/m3-sys/m3tests/src/p2/p258 Can anyone try this on a very old cm3 version, like circa 3.6/4.1/5.1?I believe I rewrote this code a few years ago... :( Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 12:01:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 10:01:54 +0000 Subject: [M3devel] m3cg right shift signed? In-Reply-To: References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com>, Message-ID: You are right, it isn't new. But I think nothing in the system was using it until the 32bit wchar support came in.I added test p257 that exercises it..so I can make sure m3cc/nt386 agree, and then update C backend to match. - Jay > Subject: Re: [M3devel] m3cg right shift signed? > From: hosking at purdue.edu > Date: Mon, 8 Jun 2015 22:00:22 -0400 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I don?t think this is a new occurrence. Surely signed right shift makes sense to retain the sign bit? > > > On Jun 8, 2015, at 9:44 PM, Jay wrote: > > > > (Not at propercomputer but sending anyway.) > > > > Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? > > > > - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 12:04:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 10:04:44 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? In-Reply-To: References: Message-ID: The bulk of this is commited now -- 4.3/4.5/4.6 removal.gcc-apple and gmp remain, for now. Any objections and I can add them back. We can also trim the frontends and libraries from 4.7 and apple. Thank you, - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 9 Jun 2015 18:30:17 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? Is there any interest in maintaining 1) gcc/m3cg/parse.c support for gcc 4.3/4.5/4.6? It is slightly messy. 1b) gcc 4.2? This is where hypothetical ARM_DARWIN support is, last I checked, years ago. 2) gcc-4.3/4.5/4.6 in-tree? Or remove them to keep checkouts smaller? There was complaint as to our source tree size, and these directories do add a lot of size for little gain. They were historically useful to transition versions. I don't think it was wrong to have the temporary growth. All targets except for ARM_DARWIN default to gcc 4.7. Do people often/ever go back and compare/debug? Or want to retain that ability, with the current ease? Or ok to get the files from history, for that rare-to-never event? If we do maintain the gcc backend, I would likely import as gcc-5.1 etc., and recreate the problem, but it can be a "rolling" problem and fix. But I'm not keen on maintaining the gcc backend anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 11:59:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:59:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? In-Reply-To: References: Message-ID: Nevermind. I fixed it. I regressed it not in the rewrite in 2010 but in a minor cleanup in 2012. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Wed, 10 Jun 2015 09:45:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? set_range appears to fail to set any bitsif requested to set one bit in a set that doesn't fit in machine word. Test case is here:https://github.com/modula3/cm3/tree/master/m3-sys/m3tests/src/p2/p258 Can anyone try this on a very old cm3 version, like circa 3.6/4.1/5.1?I believe I rewrote this code a few years ago... :( Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 23:51:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 21:51:57 +0000 Subject: [M3devel] rpath woes In-Reply-To: <557138A9.3000605@marino.st> References: <55709183.4060804@marino.st>, , <5570947F.2080600@marino.st>, <5570D114.40104@marino.st>, , <557138A9.3000605@marino.st> Message-ID: It appears FreeBSD system ld gained -z origin support in 4.7. This probably maps back to a particular GNU ld change/version. This issue bugs me. I want something simple and portable. There seems to be no perfect solution. Options include: 1 drop support for old systems 2 relink or chrpath/patchelf upon install (Apple and I think Solaris have similar utilities) 3 better predict the install path at build time I think this is what we do by default actually, just not for building "distributions" (at least make-dist.py) 4 wrapper scripts that set LD_LIBRARY_PATH; seems a popular suggestion I find anything involving wrapper scripts inelegant 5 use libtool and whatever it does, which is probably I'd guess is relink-upon-install 6 use autoconf to detect -z origin and fallback to ? $ORIGIN has the downside of possibly not working with setuid/setgid. yes: http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+10.1-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+10.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+8.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+7.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+6.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+5.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+4.7-RELEASE&arch=default&format=html no: http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+4.0-RELEASE&arch=default&format=html - Jay > Date: Fri, 5 Jun 2015 07:50:33 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > > On 6/5/2015 06:16, Jay K wrote: > > The FreeBSD4 users were surprisingly vocal surprisingly recently. > > So I put some work into it. > > I agree it is an old system. > > It's completely unsupported (for 7 years now) and they don't need the > latest M3 right? Old release still work fine, that should be enough > justification to change this on trunk. > > > > > Linux.common and FreeBSD.common look very similar but I agree different > > and if it is working for you, good, I can ignore it? > > I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? > > Was this right for FreeBSD 5 or 6 or 7 or 8? > > yes. I am using the latest binutils (thus latest ld) with M3. The base > ld is ancient and hand maintained (a GPLv3 issue) so I don't know, > Actually, the linux way is opposite: It explicitly defines SYSTEM_LD > rather than use flags for SYSTEM_CC. So rather than flags, we are > talking about the approach. > > Really nobody with a supported FreeBSD release needs to build from > scratch because it exists in ports. If anybody absolutely wants to, > just say "emulates what the port is doing". This is actually easier > said than done because in addition to using the latest binutils, there > are a number of inline substitutions that are done, see post-patch > target here, starting line 78: > > https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup > > line 85 is where newer gcc and newer as is set for m3. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 23:57:53 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 21:57:53 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556F42FF.9000106@marino.st> References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: By "gcc obsolete targets", I meant that latest gcc might not target systems that we can/do.Not that gcc versions are no longer maintained. Maybe not all that interesting. We had surprisingly recent usage on OSF/1 v4.0. I had wanted to get Irix working. And AIXand I had an older system. Anyway, I forgot about this part of it, and we could targetthem with the C backend instead of gcc. Or make current gcc work. The size is better now? - Jay > Date: Wed, 3 Jun 2015 20:10:07 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On 6/3/2015 20:04, Jay K wrote: > > If gcc obsoletes targets we want to keep, we could keep the old > > versions. Not super useful given our usage levels. > > It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not > obsolete by that definition, so all 5 of these are obsolete. > > I would definitely encourage to prune as many of these as possible. I > could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" > based on the base compiler, why are you assuming it must be built by > base compiler? > > By definition, the base compiler is only required to build base. There > are much newer and well maintained versions of gcc in openbsd ports > tree. There's no reason a ports compiler couldn't be used (I assume > this is actually common). > > > Try xz instead of gzip, maybe it halves the size? > > I can't influence github's API. It is what it is. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jun 12 07:21:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 12 Jun 2015 05:21:44 +0000 Subject: [M3devel] mklib disabled Message-ID: > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > release compiler. > 3. mklib is only used on windows targets anyway. > So, omit mklib altogether, except for windows targets. It is true it did not compile with the released non-Windows m3core. I fixed that. It is true it is only used when targeting Windows. However I want this the way it was. In particular, because w/ it disabled, I have some propensity to break Windows-specific aspects of scripts/python, that leaving it enabled reveals. It is pretty cheap. Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 12 15:39:33 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 08:39:33 -0500 Subject: [M3devel] mklib disabled In-Reply-To: References: Message-ID: <557AE115.8000009@lcwb.coop> On 06/12/2015 12:21 AM, Jay K wrote: > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > release compiler. > > 3. mklib is only used on windows targets anyway. > > So, omit mklib altogether, except for windows targets. > > > It is true it did not compile with the released non-Windows m3core. > I fixed that. > > > It is true it is only used when targeting Windows. > > > However I want this the way it was. > In particular, because w/ it disabled, I have some propensity > to break Windows-specific aspects of scripts/python, that leaving > it enabled reveals. It is pretty cheap. > > > Ok? Not sure I understand, but it sounds like you want to put the github repo in a state where it can not be built by the 5-8 release? If so, that is not OK. The github master branch should always be buildable by the most recent release. Otherwise, we are just insultingly unwelcoming to anybody who is not an insider. You are an insider. Just change it back the way you want in your working directory after you do a pull. > - Jay > > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Jun 12 16:17:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 12 Jun 2015 14:17:30 +0000 Subject: [M3devel] mklib disabled In-Reply-To: <557AE115.8000009@lcwb.coop> References: , <557AE115.8000009@lcwb.coop> Message-ID: > > It is true it did not compile with the released non-Windows m3core. > > I fixed that. > > Not sure I understand, but it sounds like you want to put the github repo > in a state where it can not be built by the 5-8 release? No. It compiles with 5.8.6 now. I fixed that. https://github.com/modula3/cm3/commit/78f21828ad80422b467e31f18c779cebc0a580eb fix for bootstrapping from previous release -- copy in everything used from m3core/winnt.i3 - Jay > Date: Fri, 12 Jun 2015 08:39:33 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] mklib disabled > > > > On 06/12/2015 12:21 AM, Jay K wrote: > > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > > release compiler. > > > 3. mklib is only used on windows targets anyway. > > > So, omit mklib altogether, except for windows targets. > > > > > > It is true it did not compile with the released non-Windows m3core. > > I fixed that. > > > > > > It is true it is only used when targeting Windows. > > > > > > However I want this the way it was. > > In particular, because w/ it disabled, I have some propensity > > to break Windows-specific aspects of scripts/python, that leaving > > it enabled reveals. It is pretty cheap. > > > > > > Ok? > > Not sure I understand, but it sounds like you want to put the github repo > in a state where it can not be built by the 5-8 release? If so, that is > not OK. The github master branch should always be buildable by the most recent > release. Otherwise, we are just insultingly unwelcoming to anybody who > is not an insider. > > You are an insider. Just change it back the way you want in your working directory > after you do a pull. > > > > - Jay > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Fri Jun 12 16:51:53 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 12 Jun 2015 16:51:53 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: References: <55774665.8060303@elstel.org>, Message-ID: <557AF209.6080408@elstel.org> Thanks a lot Rodney and Jay; that will certainly help my implementation. So far all *.mc files found on my machine have the following signature: 16_FD,00,01,{00} except a few text - .mc from PM3 which start alltogether with "begin_unit". Rodney, do you believe that I can rely on the 4th byte to be zero as generated by the Modula-3 middle end. - or would anyone be ready to uphold such a guarantee for the future? Anyone here who has applied "od" on an .mc generated by a very recent compiler? - do they start with 16_FD,10,01,?00? Most binary file types would guarantee a header of at least 4 Byte and it should be more straight forward and secure to check 32bit instead of 24bit if possible. Any suggestions? Am 10.06.15 um 02:21 schrieb Rodney M. Bates: > > > On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >> What are *.mc - files? >> They appear in TARGET - directories; >> most of them are just called _m3main.mc but some of them have other >> names. >> >> I ask because I am writing a program which should recognize and clear >> object files. >> It does not seem to be sufficient to check for uppercase directories >> which are located together with an src directory. >> >> Usually files of a specific type start with a 32bit magic; >> however the mc files all have different starting sequences. >> >> Is there still a straight forward way to recognize an .mc file just >> by its binary content? >> > > They will start with either 16_FD 16_00 16_01, produced by older > versions of cm3, > or 16_FD 16_10 16_01, produced by a very > recent head compiler. > Ignore the 4th byte. Am 09.06.15 um 22:14 schrieb Jay K: > ps: > > foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo > foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io > > again, see cm3 -keep, err better yet, cm3 -keep -verbose > You can see it running cm3cg and as and rm. > > > - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 12 19:28:57 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 12:28:57 -0500 Subject: [M3devel] mklib disabled In-Reply-To: References: , <557AE115.8000009@lcwb.coop> Message-ID: <557B16D9.2090909@lcwb.coop> On 06/12/2015 09:17 AM, Jay K wrote: > > > It is true it did not compile with the released non-Windows m3core. > > > I fixed that. > > > > Not sure I understand, but it sounds like you want to put the github repo > > in a state where it can not be built by the 5-8 release? > > > No. It compiles with 5.8.6 now. I fixed that. OK, I have no problem with that. > > https://github.com/modula3/cm3/commit/78f21828ad80422b467e31f18c779cebc0a580eb > fix for bootstrapping from previous release -- copy in everything used from m3core/winnt.i3 > > - Jay > > > > Date: Fri, 12 Jun 2015 08:39:33 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] mklib disabled > > > > > > > > On 06/12/2015 12:21 AM, Jay K wrote: > > > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > > > release compiler. > > > > 3. mklib is only used on windows targets anyway. > > > > So, omit mklib altogether, except for windows targets. > > > > > > > > > It is true it did not compile with the released non-Windows m3core. > > > I fixed that. > > > > > > > > > It is true it is only used when targeting Windows. > > > > > > > > > However I want this the way it was. > > > In particular, because w/ it disabled, I have some propensity > > > to break Windows-specific aspects of scripts/python, that leaving > > > it enabled reveals. It is pretty cheap. > > > > > > > > > Ok? > > > > Not sure I understand, but it sounds like you want to put the github repo > > in a state where it can not be built by the 5-8 release? If so, that is > > not OK. The github master branch should always be buildable by the most recent > > release. Otherwise, we are just insultingly unwelcoming to anybody who > > is not an insider. > > > > You are an insider. Just change it back the way you want in your working directory > > after you do a pull. > > > > > > > - Jay > > > > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 12 20:00:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 13:00:00 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557AF209.6080408@elstel.org> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> Message-ID: <557B1E20.7090102@lcwb.coop> On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: > > Thanks a lot Rodney and Jay; > that will certainly help my implementation. > > So far all *.mc files found on my machine have the > following signature: > 16_FD,00,01,{00} > > except a few text - .mc from PM3 which start > alltogether with "begin_unit". > > Rodney, do you believe that I can rely on the 4th byte > to be zero as generated by the Modula-3 middle end. - > or would anyone be ready to uphold such a guarantee > for the future? > The 4th byte is not really dependable for the future. It never has had a real magic number. The FD,00,01 is a version number on the binary format, so even it is likely to change. The 4th byte zero is a binary opcode for begin_unit, equivalent to the "begin_unit" in the PM3 text version. I think the most reliable long-term way is just to look for file names *.mc and *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an INTERFACE. These can be regenerated from source and will not be needed once a compile is complete, unless you are into vetting/debugging the compiler. So deleting them is quite safe. I suppose we could add a magic number. We already have a front/back end compatibility change between the release and head compilers. I can do this, if there is consensus we should. How would we choose the number? > Anyone here who has applied "od" on an .mc generated > by a very recent compiler? - do they start with > 16_FD,10,01,?00? > > Most binary file types would guarantee a header of at > least 4 Byte and it should be more straight forward and > secure to check 32bit instead of 24bit if possible. > > Any suggestions? > > > Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >> >> >> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>> What are *.mc - files? >>> They appear in TARGET - directories; >>> most of them are just called _m3main.mc but some of them have other names. >>> >>> I ask because I am writing a program which should recognize and clear object files. >>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>> >>> Usually files of a specific type start with a 32bit magic; >>> however the mc files all have different starting sequences. >>> >>> Is there still a straight forward way to recognize an .mc file just by its binary content? >>> >> >> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >> or 16_FD 16_10 16_01, produced by a very recent head compiler. >> Ignore the 4th byte. > > > Am 09.06.15 um 22:14 schrieb Jay K: >> ps: >> >> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >> >> again, see cm3 -keep, err better yet, cm3 -keep -verbose >> You can see it running cm3cg and as and rm. >> >> >> - Jay >> > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Fri Jun 12 20:51:31 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 12 Jun 2015 20:51:31 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557B1E20.7090102@lcwb.coop> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> Message-ID: <557B2A33.4010402@elstel.org> Am 12.06.15 um 20:00 schrieb Rodney M. Bates: > > > On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: >> >> Thanks a lot Rodney and Jay; >> that will certainly help my implementation. >> >> So far all *.mc files found on my machine have the >> following signature: >> 16_FD,00,01,{00} >> >> except a few text - .mc from PM3 which start >> alltogether with "begin_unit". >> >> Rodney, do you believe that I can rely on the 4th byte >> to be zero as generated by the Modula-3 middle end. - >> or would anyone be ready to uphold such a guarantee >> for the future? >> > > The 4th byte is not really dependable for the future. It never has had > a real magic number. The FD,00,01 is a version number on the binary > format, so even it is likely to change. > > The 4th byte zero is a binary opcode for begin_unit, equivalent > to the "begin_unit" in the PM3 text version. Well, the begin_unit is exactly what I check for when an .mc appears to be text. If 00 encodes begin_unit I believe it should be save to check for FD,00,01,00 and FD,10,01,00. How could an .mc file not start with begin_unit? Wouldn`t that be invalid? - or if it still would be valid I believe we didn`t generate such files, yet. - so if for the future it may start with any other command a fixed 4-byte magic which is not already interpreted would be welcome. Basically any random number should suffice as with 1.000.000 already registered file formats the probability for a clash would just be 1/4000. Nonetheless we could double- check against the database of the "file" program. Not all files have a completely random magic; f.i. pyc (compiled python files) have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be a valid date. However if we can choose things from scratch I would speak for a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers and timestamps in the following (*). It would be beneficial to have at least a cm3-middleend version number encoded since not every backend can be combined with any middle/front-end. * with a version dependent 2-byte header portion I will need a vaildly set current system date to determine whether it is a .pyc of a future version of python. > > I think the most reliable long-term way is just to look for file names > *.mc and > *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an > INTERFACE. These can be regenerated from source and will not be > needed once a > compile is complete, unless you are into vetting/debugging the compiler. > So deleting them is quite safe. Not all *.mc belong to Modula-3. I have some *.mc in my home directory which are in a fact MS Visual Studio files. That is why I prefer a combination of the file extension and file header/magic to determine whether a file can be auto- matically deleted. For Modula-3 it is also quite save to look for TARGET directories**. However if we meet a file which does not contain plain human readable text we may always want to determine in some way where the file stems from and what data it may contain. File suffixes can be stripped by accident (f.i. on an iso9660 file system) or intendedly by will. - and perhaps we do not want to look to deep into a binary before determining what it is (f.i. by a file manager). Even the "file"-tool was already reported to have a security vulnerability ... ** that will at best poorly work on a non-Unix system where file names are not case sensitive. > > I suppose we could add a magic number. We already have a front/back end > compatibility change between the release and head compilers. I can do > this, > if there is consensus we should. How would we choose the number? > > >> Anyone here who has applied "od" on an .mc generated >> by a very recent compiler? - do they start with >> 16_FD,10,01,?00? >> >> Most binary file types would guarantee a header of at >> least 4 Byte and it should be more straight forward and >> secure to check 32bit instead of 24bit if possible. >> >> Any suggestions? >> >> >> Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >>> >>> >>> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>>> What are *.mc - files? >>>> They appear in TARGET - directories; >>>> most of them are just called _m3main.mc but some of them have other >>>> names. >>>> >>>> I ask because I am writing a program which should recognize and >>>> clear object files. >>>> It does not seem to be sufficient to check for uppercase >>>> directories which are located together with an src directory. >>>> >>>> Usually files of a specific type start with a 32bit magic; >>>> however the mc files all have different starting sequences. >>>> >>>> Is there still a straight forward way to recognize an .mc file just >>>> by its binary content? >>>> >>> >>> They will start with either 16_FD 16_00 16_01, produced by older >>> versions of cm3, >>> or 16_FD 16_10 16_01, produced by a very >>> recent head compiler. >>> Ignore the 4th byte. >> >> >> Am 09.06.15 um 22:14 schrieb Jay K: >>> ps: >>> >>> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >>> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >>> >>> again, see cm3 -keep, err better yet, cm3 -keep -verbose >>> You can see it running cm3cg and as and rm. >>> >>> >>> - Jay >>> >> > From jay.krell at cornell.edu Fri Jun 12 22:10:32 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 12 Jun 2015 13:10:32 -0700 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557AF209.6080408@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> Message-ID: <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> No guarantees on any of this imho. Nor the extension. The files are usually temporary. What are the magic bytes for .c? What is the purpose here? We could add 4 ignored bytes or even a guid but it'd be a waste. - Jay On Jun 12, 2015, at 7:51 AM, Elmar Stellnberger wrote: > > Thanks a lot Rodney and Jay; > that will certainly help my implementation. > > So far all *.mc files found on my machine have the > following signature: > 16_FD,00,01,{00} > > except a few text - .mc from PM3 which start > alltogether with "begin_unit". > > Rodney, do you believe that I can rely on the 4th byte > to be zero as generated by the Modula-3 middle end. - > or would anyone be ready to uphold such a guarantee > for the future? > > Anyone here who has applied "od" on an .mc generated > by a very recent compiler? - do they start with > 16_FD,10,01,?00? > > Most binary file types would guarantee a header of at > least 4 Byte and it should be more straight forward and > secure to check 32bit instead of 24bit if possible. > > Any suggestions? > > > Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >> >> >> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>> What are *.mc - files? >>> They appear in TARGET - directories; >>> most of them are just called _m3main.mc but some of them have other names. >>> >>> I ask because I am writing a program which should recognize and clear object files. >>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>> >>> Usually files of a specific type start with a 32bit magic; >>> however the mc files all have different starting sequences. >>> >>> Is there still a straight forward way to recognize an .mc file just by its binary content? >> >> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >> or 16_FD 16_10 16_01, produced by a very recent head compiler. >> Ignore the 4th byte. > > > Am 09.06.15 um 22:14 schrieb Jay K: >> ps: >> >> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >> >> again, see cm3 -keep, err better yet, cm3 -keep -verbose >> You can see it running cm3cg and as and rm. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sat Jun 13 01:07:58 2015 From: adacore at marino.st (John Marino) Date: Sat, 13 Jun 2015 01:07:58 +0200 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: <557B664E.4040801@marino.st> On 6/10/2015 23:57, Jay K wrote: > By "gcc obsolete targets", I meant that latest gcc might not target > systems that we can/do. > Not that gcc versions are no longer maintained. > Maybe not all that interesting. > > > We had surprisingly recent usage on OSF/1 v4.0. I had wanted to get Irix > working. And AIX > and I had an older system. > > Anyway, I forgot about this part of it, and we could target > them with the C backend instead of gcc. Or make current gcc work. > > > The size is better now? Yes. It reduced the distribution file from Github from 150Mb to 91Mb, a significant reduction of 59Mb (almost 40%) Thanks, John From estellnb at elstel.org Sat Jun 13 10:40:40 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Sat, 13 Jun 2015 10:40:40 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> Message-ID: <557BEC88.6020608@elstel.org> Am 12.06.15 um 22:10 schrieb Jay: > No guarantees on any of this imho. Nor the extension. The files are > usually temporary. What are the magic bytes for .c? What is the > purpose here? We could add 4 ignored bytes or even a guid but it'd be > a waste. human readable text files do not need any magic but binaries do (though auto-generated text files by a program will need a header, too). A cm3 and middle end version numbers for a compatibilty check if not also a creation timestamp after the magic would be very beneficial, too. It is unprofessional and a bad habit to emit a binary stream without a header and versioning information. Have you never used a ready compiled cm3cg for compiling a new frontend? - There should be a middle end version number to check whether this is prone to fail (and possibly one trying to force application of cm3cg on a newer intermediate code stream. ). -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jun 13 13:37:43 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Jun 2015 07:37:43 -0400 Subject: [M3devel] On magic numbers In-Reply-To: <557B2A33.4010402@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> Message-ID: <20150613113743.GB23865@topoi.pooq.com> On Fri, Jun 12, 2015 at 08:51:31PM +0200, Elmar Stellnberger wrote: > Basically any random > number should suffice as with 1.000.000 already registered file formats the > probability for a clash would just be 1/4000. Nonetheless we could double- > check against the database of the "file" program. For more collision-freeness for the foreseeable future, I'd suggest a 64-bit random number. Even if there were a collision with someone else's 32-bit number, then next 32 bits would likely resolve the issue. It's not too far-fetched to assume that the number of different file formats will continue increasing exponentially even as our world-wide data storage increases. And maybe it's tie that the hash codes we use for data types also increase in length. I've always considered 32 bits a bit too small for this, especially in the days of *huge* program libraries. Maybe a necessary evil as a concession to antiquated linkers, but it could legitimately be made platform-dependent. For backward copatibility, the compiler could just start checking for the magic number. If it's present, skip it. If it's absent, go on as at present. > Not all files have a completely random magic; f.i. pyc (compiled > python files) > have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be > a valid date. However if we can choose things from scratch I would speak for > a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers > and timestamps in the following (*). > It would be beneficial to have at least a cm3-middleend version number > encoded since not every backend can be combined with any middle/front-end. Of course this should still be appended to the 128 (or however many) bits. -- hendrik From hendrik at topoi.pooq.com Sat Jun 13 13:40:59 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Jun 2015 07:40:59 -0400 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557BEC88.6020608@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> <557BEC88.6020608@elstel.org> Message-ID: <20150613114059.GC23865@topoi.pooq.com> On Sat, Jun 13, 2015 at 10:40:40AM +0200, Elmar Stellnberger wrote: > Am 12.06.15 um 22:10 schrieb Jay: > >No guarantees on any of this imho. Nor the extension. The files > >are usually temporary. What are the magic bytes for .c? What is > >the purpose here? We could add 4 ignored bytes or even a guid but > >it'd be a waste. > human readable text files do not need any magic > but binaries do (though auto-generated text files > by a program will need a header, too). > > A cm3 and middle end version numbers for a > compatibilty check if not also a creation timestamp > after the magic would be very beneficial, too. > > It is unprofessional and a bad habit to emit a binary > stream without a header and versioning information. > Have you never used a ready compiled cm3cg for > compiling a new frontend? - There should be a > middle end version number to check whether this > is prone to fail (and possibly one trying to force > application of cm3cg on a newer intermediate code > stream. ). Not to mention the possibility of having a compatibility mode for an old file format, in case it's not too much work. -- hendrik From rodney_bates at lcwb.coop Sat Jun 13 20:23:20 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:23:20 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557B2A33.4010402@elstel.org> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> Message-ID: <557C7518.6000700@lcwb.coop> On 06/12/2015 01:51 PM, Elmar Stellnberger wrote: > Am 12.06.15 um 20:00 schrieb Rodney M. Bates: >> >> >> On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: >>> >>> Thanks a lot Rodney and Jay; >>> that will certainly help my implementation. >>> >>> So far all *.mc files found on my machine have the >>> following signature: >>> 16_FD,00,01,{00} >>> >>> except a few text - .mc from PM3 which start >>> alltogether with "begin_unit". >>> >>> Rodney, do you believe that I can rely on the 4th byte >>> to be zero as generated by the Modula-3 middle end. - >>> or would anyone be ready to uphold such a guarantee >>> for the future? >>> I doubt anyone would want to guarantee any of these 4 bytes, as they are actual content information and not a magic number. However, I would say the 1st (FD) and last (00, code for begin_unit) have pretty low probability of changing. The middle two contain a version number, so can be expected to change. The second (counting from one) is least significant, and more likely to change. We already have two possible values here, as I reported. >> >> The 4th byte is not really dependable for the future. It never has had >> a real magic number. The FD,00,01 is a version number on the binary >> format, so even it is likely to change. >> >> The 4th byte zero is a binary opcode for begin_unit, equivalent >> to the "begin_unit" in the PM3 text version. > Well, the begin_unit is exactly what I check for when an .mc appears to be text. > If 00 encodes begin_unit I believe it should be save to check for FD,00,01,00 > and FD,10,01,00. How could an .mc file not start with begin_unit? Wouldn`t > that be invalid? - or if it still would be valid I believe we didn`t generate such > files, yet. > - so if for the future it may start with any other command a fixed 4-byte magic > which is not already interpreted would be welcome. Basically any random > number should suffice as with 1.000.000 already registered file formats the > probability for a clash would just be 1/4000. Nonetheless we could double- > check against the database of the "file" program. > Not all files have a completely random magic; f.i. pyc (compiled python files) > have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be > a valid date. However if we can choose things from scratch I would speak for > a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers > and timestamps in the following (*). > It would be beneficial to have at least a cm3-middleend version number > encoded since not every backend can be combined with any middle/front-end. > > * with a version dependent 2-byte header portion I will need a vaildly set current > system date to determine whether it is a .pyc of a future version of python. > >> >> I think the most reliable long-term way is just to look for file names *.mc and >> *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an >> INTERFACE. These can be regenerated from source and will not be needed once a >> compile is complete, unless you are into vetting/debugging the compiler. >> So deleting them is quite safe. > Not all *.mc belong to Modula-3. I have some *.mc in my home directory which > are in a fact MS Visual Studio files. That is why I prefer a combination of the > file extension and file header/magic to determine whether a file can be auto- > matically deleted. OK, file names are not adequate. > For Modula-3 it is also quite save to look for TARGET directories**. However if we > meet a file which does not contain plain human readable text we may always > want to determine in some way where the file stems from and what data it may > contain. File suffixes can be stripped by accident (f.i. on an iso9660 file system) > or intendedly by will. - and perhaps we do not want to look to deep into a > binary before determining what it is (f.i. by a file manager). Even the "file"-tool > was already reported to have a security vulnerability ... > > ** that will at best poorly work on a non-Unix system where file names are not > case sensitive. > >> >> I suppose we could add a magic number. We already have a front/back end >> compatibility change between the release and head compilers. I can do this, >> if there is consensus we should. How would we choose the number? >> >> >>> Anyone here who has applied "od" on an .mc generated >>> by a very recent compiler? - do they start with >>> 16_FD,10,01,?00? >>> >>> Most binary file types would guarantee a header of at >>> least 4 Byte and it should be more straight forward and >>> secure to check 32bit instead of 24bit if possible. >>> >>> Any suggestions? >>> >>> >>> Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >>>> >>>> >>>> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>>>> What are *.mc - files? >>>>> They appear in TARGET - directories; >>>>> most of them are just called _m3main.mc but some of them have other names. >>>>> >>>>> I ask because I am writing a program which should recognize and clear object files. >>>>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>>>> >>>>> Usually files of a specific type start with a 32bit magic; >>>>> however the mc files all have different starting sequences. >>>>> >>>>> Is there still a straight forward way to recognize an .mc file just by its binary content? >>>>> >>>> >>>> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >>>> or 16_FD 16_10 16_01, produced by a very recent head compiler. >>>> Ignore the 4th byte. >>> >>> >>> Am 09.06.15 um 22:14 schrieb Jay K: >>>> ps: >>>> >>>> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >>>> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >>>> >>>> again, see cm3 -keep, err better yet, cm3 -keep -verbose >>>> You can see it running cm3cg and as and rm. >>>> >>>> >>>> - Jay >>>> >>> >> > > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Jun 13 20:27:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:27:00 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557BEC88.6020608@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> <557BEC88.6020608@elstel.org> Message-ID: <557C75F4.4000206@lcwb.coop> On 06/13/2015 03:40 AM, Elmar Stellnberger wrote: > Am 12.06.15 um 22:10 schrieb Jay: >> No guarantees on any of this imho. Nor the extension. The files are usually temporary. What are the magic bytes for .c? What is the purpose here? We could add 4 ignored bytes or even a guid but it'd be a waste. > human readable text files do not need any magic > but binaries do (though auto-generated text files > by a program will need a header, too). > > A cm3 and middle end version numbers for a > compatibilty check if not also a creation timestamp > after the magic would be very beneficial, too. > > It is unprofessional and a bad habit to emit a binary > stream without a header and versioning information. > Have you never used a ready compiled cm3cg for > compiling a new frontend? - There should be a > middle end version number to check whether this > is prone to fail (and possibly one trying to force > application of cm3cg on a newer intermediate code > stream. ). Actually, we have had such a version number, from very early. But it had never changed, until recently, when I changed it to detect this very problem. Now, if you use mismatched cm3cg, you will get an informative error message, instead of something similar to "unrecognized op code" (I paraphrased that.) > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Jun 13 20:42:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:42:26 -0500 Subject: [M3devel] On magic numbers In-Reply-To: <20150613113743.GB23865@topoi.pooq.com> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> <20150613113743.GB23865@topoi.pooq.com> Message-ID: <557C7992.4040103@lcwb.coop> On 06/13/2015 06:37 AM, Hendrik Boom wrote: > On Fri, Jun 12, 2015 at 08:51:31PM +0200, Elmar Stellnberger wrote: > >> Basically any random >> number should suffice as with 1.000.000 already registered file formats the >> probability for a clash would just be 1/4000. Nonetheless we could double- >> check against the database of the "file" program. > > For more collision-freeness for the foreseeable future, I'd suggest a > 64-bit random number. Even if there were a collision with someone > else's 32-bit number, then next 32 bits would likely resolve the issue. > There already is a 64-bit "signature" hash of type structure computed in the compiler, but it is only used in pickles. Elsewhere, a 32-bit "UID" is used. It is just the XOR of the halves of the signature. It was very difficult for me to ferret this conclusion out of the code. The UID is also called something else (I don't remember what) in m3linker and the messages it emits, and may even have more names, making for confusing error messages and difficulty understanding the build system. The fingerprint algorithm has comments suggesting a careful design by someone familiar with high-quality hashes. (That's not me!) Using the full 64 bits everywhere would probably create some annoying transitional compatibility problems. > It's not too far-fetched to assume that the number of different file > formats will continue increasing exponentially even as our world-wide > data storage increases. > > And maybe it's tie that the hash codes we use for data types also > increase in length. I've always considered 32 bits a bit too small for > this, especially in the days of *huge* program libraries. Maybe a > necessary evil as a concession to antiquated linkers, but it could > legitimately be made platform-dependent. > > For backward copatibility, the compiler could just start checking for > the magic number. If it's present, skip it. If it's absent, go on as > at present. > >> Not all files have a completely random magic; f.i. pyc (compiled >> python files) >> have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be >> a valid date. However if we can choose things from scratch I would speak for >> a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers >> and timestamps in the following (*). >> It would be beneficial to have at least a cm3-middleend version number >> encoded since not every backend can be combined with any middle/front-end. > > Of course this should still be appended to the 128 (or however many) > bits. > > -- hendrik > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Jun 14 03:45:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 14 Jun 2015 01:45:19 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <20150613114059.GC23865@topoi.pooq.com> References: <55774665.8060303@elstel.org>, , , <557AF209.6080408@elstel.org>, <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com>, <557BEC88.6020608@elstel.org>, <20150613114059.GC23865@topoi.pooq.com> Message-ID: You should never mix versions. There is no need for a compatibilty mode. You get compatibility mode by building/using an older cm3 and cm3cg together. The system is meant to be tightly coupled. The file format exists I believe primarily as a licensing artifact. When you use the C backend, or the NT386 backend, or presumably LLVM backend, you don't get the files. An approximation of them is used in-memory for the C backend and the NT386 backend never materializes the data, they are just temporary call frames, one at a time. Unless you are in a development mode. Then m3cgcat is a useful dumper of them, and it can translate to C calling into the C backend.. The files are not meant to live long. And even if they did -- hypothetical cm3cginterp, cm3cjit, cm3cgrun -- it'd still be tightly coupled. We are free to change enum values and add/remove fields to each enum. The system has changed through time and there is no need to lock these aspects down. What we need to be compatible with it .m3 and .i3 files. They are currency. There is perhaps some need to be compatible with .o/.mo/.io and even .m3.c/.i3.c. The fact that the C backend has a different ABI than the gcc backend, in particular because of nested functions taking the frame pointer as the last C parameter and not in a special register, worries me, and makes me err toward appending "c" to BUILD_DIR and such, and maybe even TARGET. or "csj" for setjmp -- so that later we have "cppeh" C++ with C++ exception handling.. But anyway, imho .mc files are internal, assuming Tony agrees.. - Jay > Date: Sat, 13 Jun 2015 07:40:59 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3: what are *.mc files > > On Sat, Jun 13, 2015 at 10:40:40AM +0200, Elmar Stellnberger wrote: > > (and possibly one trying to force > > application of cm3cg on a newer intermediate code > > stream. ). > > Not to mention the possibility of having a compatibility mode for an > old file format, in case it's not too much work. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 16 10:43:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 16 Jun 2015 08:43:10 +0000 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu> References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu> Message-ID: I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. From: hosking at cs.purdue.edu Date: Thu, 5 Sep 2013 12:05:07 -0400 To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] TextLiteral.MaxBytes again Here is a modest proposal that should resolve this issue. Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). Updated TextLiteral.i3 and TextLiteral.m3 attached. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 16 10:43:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 16 Jun 2015 08:43:42 +0000 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu>, Message-ID: er..the proposal was old, but nevertheless.. From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: RE: [M3devel] TextLiteral.MaxBytes again Date: Tue, 16 Jun 2015 08:43:10 +0000 I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. From: hosking at cs.purdue.edu Date: Thu, 5 Sep 2013 12:05:07 -0400 To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] TextLiteral.MaxBytes again Here is a modest proposal that should resolve this issue. Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). Updated TextLiteral.i3 and TextLiteral.m3 attached. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 17 18:18:15 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 17 Jun 2015 11:18:15 -0500 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu>, Message-ID: <55819DC7.4090601@lcwb.coop> On 06/16/2015 03:43 AM, Jay K wrote: > er..the proposal was old, but nevertheless.. > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] TextLiteral.MaxBytes again > Date: Tue, 16 Jun 2015 08:43:10 +0000 > > I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. > > If you do this, there are a number of places that should be reviewed for fallout. I think something in runtime has some hard-coded value, and several places there and in pickles use TextLiteral. Plus probably a few tests around. Actually, these should probably be looked at even with the latest change, tho' the latter would not require introduction of unsafe code. > > > From: hosking at cs.purdue.edu > Date: Thu, 5 Sep 2013 12:05:07 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] TextLiteral.MaxBytes again > > Here is a modest proposal that should resolve this issue. > Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; > Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). > > Updated TextLiteral.i3 and TextLiteral.m3 attached. > > > > Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, > tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference > type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 19 03:28:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 18 Jun 2015 20:28:43 -0500 Subject: [M3devel] Anybody using Rd.UnGetCharMulti? Message-ID: <5583704B.1040006@lcwb.coop> Does anybody have code using Rd.UnGetCharMulti? I want to change it a bit to unget several characters in a single call. Some existing calls would have to be changed slightly. -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Mon Jun 22 04:44:20 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 21 Jun 2015 19:44:20 -0700 Subject: [M3devel] ARM_LINUX Bootstrap Compiler Message-ID: Does anyone have a working ARM_LINUX compiler they'd care to post? From the list it does seem some people had it running. If so and you need somewhere to upload it https://github.com/modula3/cm3/releases is probably a good place. - Darko -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Jun 22 10:21:42 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 22 Jun 2015 01:21:42 -0700 Subject: [M3devel] ARM_LINUX Bootstrap Compiler In-Reply-To: References: Message-ID: <20150622082142.E7A8A1A2066@async.async.caltech.edu> I do have it for Raspberry Pi and BeagleBone Black. Works great. But I'll have to look for it. Mika Darko Volaric writes: >--047d7b4140882fa505051912409b >Content-Type: text/plain; charset=UTF-8 > >Does anyone have a working ARM_LINUX compiler they'd care to post? From the >list it does seem some people had it running. > >If so and you need somewhere to upload it >https://github.com/modula3/cm3/releases is probably a good place. > >- Darko > >--047d7b4140882fa505051912409b >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Does anyone have a working=C2=A0ARM_LINUX compiler they= >9;d care to post? From the list it does seem some people had it running.v>
If so and you need somewhere to upload it =C2=A0"https://github.com/modula3/cm3/releases">https://github.com/modula3/cm3/re= >leases =C2=A0is probably a good place.

- Darko= >

> >--047d7b4140882fa505051912409b-- From jay.krell at cornell.edu Mon Jun 22 18:07:55 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 22 Jun 2015 16:07:55 +0000 Subject: [M3devel] ARM_LINUX Bootstrap Compiler In-Reply-To: <20150622082142.E7A8A1A2066@async.async.caltech.edu> References: , <20150622082142.E7A8A1A2066@async.async.caltech.edu> Message-ID: Please try the C backend? Like this: Make sure *target* system has working cc/as/ld/make. cd cm3/scripts/python ./boot1.py ARM_LINUX c You will get something like: cm3-boot-ARM_LINUX-d5.10.0-20150622.tar.gz That works for me. The tail will be replaced by time/date. Copy that to the target, extract, cd, make. See if ./cm3 then works. It should print an error, like: -- building in I386_DARWIN --- Fatal Error: duplicate unit: /cm3/pkg/m3core/src/C/Common/CerrnoC.c ../CerrnoC.c and then, like, have Python installed, and: mkdir /usr/local/cm3/bin mv cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH scripts/python/boot2.sh boot2.sh builds everything. If target system doesn't have cc/as/ld/make, then you cantry using cross tools. Alter $PATH or edit the top of the generated Makefile. And then you can use make-dist.py to build (again) and package everythingand upload that? Thanks, - Jay > To: lists at darko.org > Date: Mon, 22 Jun 2015 01:21:42 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] ARM_LINUX Bootstrap Compiler > > I do have it for Raspberry Pi and BeagleBone Black. Works great. > > But I'll have to look for it. > > Mika > > Darko Volaric writes: > >--047d7b4140882fa505051912409b > >Content-Type: text/plain; charset=UTF-8 > > > >Does anyone have a working ARM_LINUX compiler they'd care to post? From the > >list it does seem some people had it running. > > > >If so and you need somewhere to upload it > >https://github.com/modula3/cm3/releases is probably a good place. > > > >- Darko > > > >--047d7b4140882fa505051912409b > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Does anyone have a working=C2=A0ARM_LINUX compiler they= > >9;d care to post? From the list it does seem some people had it running. >v>
If so and you need somewhere to upload it =C2=A0 >"https://github.com/modula3/cm3/releases">https://github.com/modula3/cm3/re= > >leases =C2=A0is probably a good place.

- Darko= > >

> > > >--047d7b4140882fa505051912409b-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 23 03:50:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 23 Jun 2015 01:50:21 +0000 Subject: [M3devel] pthread assertion failure exiting formsedit Message-ID: Every time I exit formsedit on MacOS X: (gdb) runStarting program: /cm3/bin/formsedit Reading symbols for shared libraries +++++++++++++++++++++..... done ****** runtime error:*** <*ASSERT*> failed.*** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216*** Program received signal SIGABRT, Aborted.0x940181ae in semaphore_wait_signal_trap ()(gdb) bt#0 0x940181ae in semaphore_wait_signal_trap ()#1 0x9404a1c6 in _pthread_cond_wait ()#2 0x9408f449 in pthread_cond_wait ()#3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459#4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239#5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581#6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593#7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52#8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62#9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408#10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115#11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; LOOP IF alertable AND self.alerted THEN self.alerted := FALSE; <*ASSERT self.waitingOn = c.mutex*> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Jun 23 10:41:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 23 Jun 2015 01:41:17 -0700 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: References: Message-ID: <20150623084117.468E41A2069@async.async.caltech.edu> Has anyone run the thread tester with the arguments I specified a few weeks ago on this OS/compiler version? Jay K writes: >--_cbc4bf8e-1039-413b-b25f-4e44135e099e_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Every time I exit formsedit on MacOS X: > >(gdb) runStarting program: /cm3/bin/formsedit Reading symbols for shared li= >braries +++++++++++++++++++++..... done > >****** runtime error:*** <*ASSERT*> failed.*** file "../src/thread/PT= >HREAD/ThreadPThread.m3"=2C line 216*** From rodney_bates at lcwb.coop Fri Jun 26 18:03:53 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 26 Jun 2015 11:03:53 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: References: Message-ID: <558D77E9.9040403@lcwb.coop> This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed on this target. I am rerunning it, but it has gone all night and is still going. self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to top of stack--confusing, to me) looks irrelevant. *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 *** Program received signal SIGABRT, Aborted. [Switching to Thread 140370469975808 (LWP 17456)] 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) up #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) up #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) Current language: auto; currently c (m3gdb) up #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 20 Cstdlib.abort (); Current language: auto; currently Modula-3 (m3gdb) bt #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 #7 0x00007faaa4bad577 in Crash (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) at ../src/runtime/common/RTException.m3:79 #8 0x00007faaa4bad26f in DefaultBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 #9 0x00007faaa4bad1bf in InvokeBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 #10 0x00007faaa4bbad0a in Raise (act= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 #11 0x00007faaa4bad31c in DefaultBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 #12 0x00007faaa4bad1bf in InvokeBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 #13 0x00007faaa4bbad0a in Raise (act= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) at ../src/thread/PTHREAD/ThreadPThread.m3:216 #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) frame 16 #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) at ../src/thread/PTHREAD/ThreadPThread.m3:216 216 <*ASSERT self.waitingOn = c.mutex*> (m3gdb) info arg self = 16_0000000001856130 m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END alertable = TRUE (m3gdb) info loc prev = 16_00007faa8bffeb90 next = 16_00007faaa4bc1db6 (m3gdb) p self^ $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; END; END; END (m3gdb) p c.name $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 (m3gdb) p self.waitingOn $15 = NIL (m3gdb) p c.mutex $16 = 16_00007faa84000e20 (m3gdb) On a previous run, after more poking around in the debugger, I had: (m3gdb) p c.name $12 = (*16_0000000000615b90*) "all editors closed" on 06/22/2015 08:50 PM, Jay K wrote: > Every time I exit formsedit on MacOS X: > > > (gdb) run > Starting program: /cm3/bin/formsedit > Reading symbols for shared libraries +++++++++++++++++++++..... done > > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 > *** > > > Program received signal SIGABRT, Aborted. > 0x940181ae in semaphore_wait_signal_trap () > (gdb) bt > #0 0x940181ae in semaphore_wait_signal_trap () > #1 0x9404a1c6 in _pthread_cond_wait () > #2 0x9408f449 in pthread_cond_wait () > #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 > #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 > #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 > #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 > #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 > #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 > #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 > #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 > #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 > > > WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; > LOOP > IF alertable AND self.alerted THEN > self.alerted := FALSE; > <*ASSERT self.waitingOn = c.mutex*> > > NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. > > > - Jay > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 04:20:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 27 Jun 2015 21:20:34 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: <558D77E9.9040403@lcwb.coop> References: <558D77E9.9040403@lcwb.coop> Message-ID: <558F59F2.8060406@lcwb.coop> Tony, please review/comment on my analysis, as this kind if code is very delicate, and this is the first time I have looked at it. This is in ThreadPThread.m3. It is possible that thread th, waiting on an M3 Condition variable in XWait, line 239, by actually waiting on pthreads condition variable self.cond, can be both Signaled and Alerted, before it reacquires self.mutex. Either of Signal or Alert will pthread_cond_signal self.cond, but the other can acquire self.mutex before th reacquires it, after the first pthread_cond_signal. When th finally gets self.mutex, both self.alerted and self.waitingOn=NIL. I propose to wrap lines 216 thru' 227 inside IF self.waitingOn # NIL THEN ... Presumably, if both Signal and Alert have happened, we want to raise Alerted, rather than just wake up th. On 06/26/2015 11:03 AM, Rodney M. Bates wrote: > This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed > on this target. I am rerunning it, but it has gone all night and is still going. > > self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be > diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to > top of stack--confusing, to me) looks irrelevant. > > > > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 > *** > > > Program received signal SIGABRT, Aborted. > [Switching to Thread 140370469975808 (LWP 17456)] > 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) up > #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) up > #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 > 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) > Current language: auto; currently c > (m3gdb) up > #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 > 20 Cstdlib.abort (); > Current language: auto; currently Modula-3 > (m3gdb) bt > #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 > #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 > #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 > #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 > #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, > msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 > #7 0x00007faaa4bad577 in Crash (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) > at ../src/runtime/common/RTException.m3:79 > #8 0x00007faaa4bad26f in DefaultBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 > #9 0x00007faaa4bad1bf in InvokeBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 > #10 0x00007faaa4bbad0a in Raise (act= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 > #11 0x00007faaa4bad31c in DefaultBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 > #12 0x00007faaa4bad1bf in InvokeBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 > #13 0x00007faaa4bbad0a in Raise (act= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 > #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 > #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) > at ../src/thread/PTHREAD/ThreadPThread.m3:216 > #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 > #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 > #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 > #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 > #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 > #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) frame 16 > #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) > at ../src/thread/PTHREAD/ThreadPThread.m3:216 > 216 <*ASSERT self.waitingOn = c.mutex*> > (m3gdb) info arg > self = 16_0000000001856130 > m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END > c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END > alertable = TRUE > (m3gdb) info loc > prev = 16_00007faa8bffeb90 > next = 16_00007faaa4bc1db6 > (m3gdb) p self^ > $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; > nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; > stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, > Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; > pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; > END; END; END > (m3gdb) p c.name > $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 > (m3gdb) p self.waitingOn > $15 = NIL > (m3gdb) p c.mutex > $16 = 16_00007faa84000e20 > (m3gdb) > > > On a previous run, after more poking around in the debugger, I had: > > (m3gdb) p c.name > $12 = (*16_0000000000615b90*) "all editors closed" > > on 06/22/2015 08:50 PM, Jay K wrote: > > >> Every time I exit formsedit on MacOS X: >> >> >> (gdb) run >> Starting program: /cm3/bin/formsedit >> Reading symbols for shared libraries +++++++++++++++++++++..... done >> >> >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >> *** >> >> >> Program received signal SIGABRT, Aborted. >> 0x940181ae in semaphore_wait_signal_trap () >> (gdb) bt >> #0 0x940181ae in semaphore_wait_signal_trap () >> #1 0x9404a1c6 in _pthread_cond_wait () >> #2 0x9408f449 in pthread_cond_wait () >> #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 >> #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 >> #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 >> #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 >> #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 >> #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 >> #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 >> #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 >> #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 >> >> >> WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; >> LOOP >> IF alertable AND self.alerted THEN >> self.alerted := FALSE; >> <*ASSERT self.waitingOn = c.mutex*> >> >> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. >> >> >> - Jay >> > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 04:29:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 27 Jun 2015 21:29:41 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: <558F59F2.8060406@lcwb.coop> References: <558D77E9.9040403@lcwb.coop> <558F59F2.8060406@lcwb.coop> Message-ID: <558F5C15.4050303@lcwb.coop> I tried this locally (including lines 228 & 229 also) and it does make the current symptom go away. Also, the thread test is not showing any worrisome output, after a few minutes. BTW, I take it this test is not intended to terminate on its own? On 06/27/2015 09:20 PM, Rodney M. Bates wrote: > Tony, please review/comment on my analysis, as this kind if code is very delicate, > and this is the first time I have looked at it. > > This is in ThreadPThread.m3. > > It is possible that thread th, waiting on an M3 Condition variable in XWait, > line 239, by actually waiting on pthreads condition variable self.cond, > can be both Signaled and Alerted, before it reacquires self.mutex. Either of > Signal or Alert will pthread_cond_signal self.cond, but the other can acquire > self.mutex before th reacquires it, after the first pthread_cond_signal. > When th finally gets self.mutex, both self.alerted and self.waitingOn=NIL. > > I propose to wrap lines 216 thru' 227 inside IF self.waitingOn # NIL THEN ... > > Presumably, if both Signal and Alert have happened, we want to raise Alerted, > rather than just wake up th. > > On 06/26/2015 11:03 AM, Rodney M. Bates wrote: >> This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed >> on this target. I am rerunning it, but it has gone all night and is still going. >> >> self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be >> diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to >> top of stack--confusing, to me) looks irrelevant. >> >> >> >> >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >> *** >> >> >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 140370469975808 (LWP 17456)] >> 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) up >> #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) up >> #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 >> 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) >> Current language: auto; currently c >> (m3gdb) up >> #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 >> 20 Cstdlib.abort (); >> Current language: auto; currently Modula-3 >> (m3gdb) bt >> #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 >> #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 >> #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 >> #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 >> #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, >> msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 >> #7 0x00007faaa4bad577 in Crash (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) >> at ../src/runtime/common/RTException.m3:79 >> #8 0x00007faaa4bad26f in DefaultBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 >> #9 0x00007faaa4bad1bf in InvokeBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 >> #10 0x00007faaa4bbad0a in Raise (act= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 >> #11 0x00007faaa4bad31c in DefaultBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 >> #12 0x00007faaa4bad1bf in InvokeBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 >> #13 0x00007faaa4bbad0a in Raise (act= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 >> #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 >> #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) >> at ../src/thread/PTHREAD/ThreadPThread.m3:216 >> #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 >> #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 >> #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 >> #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 >> #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 >> #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) frame 16 >> #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) >> at ../src/thread/PTHREAD/ThreadPThread.m3:216 >> 216 <*ASSERT self.waitingOn = c.mutex*> >> (m3gdb) info arg >> self = 16_0000000001856130 >> m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END >> c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END >> alertable = TRUE >> (m3gdb) info loc >> prev = 16_00007faa8bffeb90 >> next = 16_00007faaa4bc1db6 >> (m3gdb) p self^ >> $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; >> nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; >> stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, >> Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; >> pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; >> END; END; END >> (m3gdb) p c.name >> $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 >> (m3gdb) p self.waitingOn >> $15 = NIL >> (m3gdb) p c.mutex >> $16 = 16_00007faa84000e20 >> (m3gdb) >> >> >> On a previous run, after more poking around in the debugger, I had: >> >> (m3gdb) p c.name >> $12 = (*16_0000000000615b90*) "all editors closed" >> >> on 06/22/2015 08:50 PM, Jay K wrote: >> >> >>> Every time I exit formsedit on MacOS X: >>> >>> >>> (gdb) run >>> Starting program: /cm3/bin/formsedit >>> Reading symbols for shared libraries +++++++++++++++++++++..... done >>> >>> >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >>> *** >>> >>> >>> Program received signal SIGABRT, Aborted. >>> 0x940181ae in semaphore_wait_signal_trap () >>> (gdb) bt >>> #0 0x940181ae in semaphore_wait_signal_trap () >>> #1 0x9404a1c6 in _pthread_cond_wait () >>> #2 0x9408f449 in pthread_cond_wait () >>> #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 >>> #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 >>> #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 >>> #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 >>> #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 >>> #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 >>> #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 >>> #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 >>> #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 >>> >>> >>> WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; >>> LOOP >>> IF alertable AND self.alerted THEN >>> self.alerted := FALSE; >>> <*ASSERT self.waitingOn = c.mutex*> >>> >>> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. >>> >>> >>> - Jay >>> >> > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 19:58:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 28 Jun 2015 12:58:51 -0500 Subject: [M3devel] pthread scheduling fairness of Signal Message-ID: <559035DB.3040101@lcwb.coop> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really what we want? -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 29 15:40:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 29 Jun 2015 08:40:48 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: References: <559035DB.3040101@lcwb.coop> Message-ID: <55914AE0.8080407@lcwb.coop> On 06/28/2015 01:16 PM, Antony Hosking wrote: > It mirrors the user-level threads information as far as I recall. > Do you think that was intentional in the original user-level implementation? It seems like an open invitation to starvation. And it seems even more important for condition variables than mutexen. (hush, spell checker!) Mika's high-volume thread tester continuously churns out "Thread appears starved or deadlocked" messages by the hundreds. > Sent from my iPad > >> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: >> >> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I >> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those >> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really >> what we want? >> -- >> Rodney Bates >> rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From mika at async.caltech.edu Mon Jun 29 17:31:44 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 29 Jun 2015 08:31:44 -0700 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <55914AE0.8080407@lcwb.coop> References: <559035DB.3040101@lcwb.coop> <55914AE0.8080407@lcwb.coop> Message-ID: <20150629153144.B49851A2062@async.async.caltech.edu> Well code shouldn't depend on fairness. I don't think it's a major issue if it's backwards. I didn't add the warning messages about starved threads to the thread tester. "Rodney M. Bates" writes: > > >On 06/28/2015 01:16 PM, Antony Hosking wrote: >> It mirrors the user-level threads information as far as I recall. >> > >Do you think that was intentional in the original user-level implementation? >It seems like an open invitation to starvation. And it seems even more >important for condition variables than mutexen. (hush, spell checker!) > >Mika's high-volume thread tester continuously churns out >"Thread appears starved or deadlocked" messages by the hundreds. > >> Sent from my iPad >> >>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote >: >>> >>> In vetting ThreadPThread in conjunction with the formsedit assertion failur >e, I >>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but >those >>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this r >eally >>> what we want? >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >> > >-- >Rodney Bates >rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 30 02:18:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 29 Jun 2015 19:18:41 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu> References: <559035DB.3040101@lcwb.coop> <55914AE0.8080407@lcwb.coop> <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu> Message-ID: <5591E061.8090400@lcwb.coop> On 06/29/2015 08:53 AM, Antony Hosking wrote: > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: >> >> >> >> On 06/28/2015 01:16 PM, Antony Hosking wrote: >>> It mirrors the user-level threads information as far as I recall. >>> >> >> Do you think that was intentional in the original user-level implementation? >> It seems like an open invitation to starvation. And it seems even more >> important for condition variables than mutexen. (hush, spell checker!) > > Perhaps take a look at the original user-level implementation to confirm. > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening Mutex waiters and LIFO for Condition waiters. Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. (n=# of waiting threads.) I think n will most often be quite small, but occasionally, maybe not, and if/when high thread counts do occur, it would be a good time for things to be fast. It would be quite easy to make both cases FIFO and O(1). No synchronization changes, just change a few lines of code while holding the same locks that are already being held. If nobody objects, I think I will try this. >> >> Mika's high-volume thread tester continuously churns out >> "Thread appears starved or deadlocked" messages by the hundreds. >> >>> Sent from my iPad >>> >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: >>>> >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really >>>> what we want? >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 30 08:37:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 30 Jun 2015 06:37:50 +0000 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <5591E061.8090400@lcwb.coop> References: <559035DB.3040101@lcwb.coop>, , <55914AE0.8080407@lcwb.coop>, <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu>, <5591E061.8090400@lcwb.coop> Message-ID: 1) This sounds good. 2) Can you please apply your analysis to the Win32 version also? Tangentially, I've said both of these before: 3) I'd really like to layer the pthread and Win32 versions over a common interface. 4) Possibly make it much thinner, if Modula-3 semantics are close enough to pthread/win32+cv. - Jay > Date: Mon, 29 Jun 2015 19:18:41 -0500 > From: rodney_bates at lcwb.coop > To: hosking at purdue.edu; mika at async.caltech.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] pthread scheduling fairness of Signal > > > > On 06/29/2015 08:53 AM, Antony Hosking wrote: > > > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: > >> > >> > >> > >> On 06/28/2015 01:16 PM, Antony Hosking wrote: > >>> It mirrors the user-level threads information as far as I recall. > >>> > >> > >> Do you think that was intentional in the original user-level implementation? > >> It seems like an open invitation to starvation. And it seems even more > >> important for condition variables than mutexen. (hush, spell checker!) > > > > Perhaps take a look at the original user-level implementation to confirm. > > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > > > > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening > Mutex waiters and LIFO for Condition waiters. > > Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. > (n=# of waiting threads.) I think n will most often be quite small, but occasionally, > maybe not, and if/when high thread counts do occur, it would be a good time for things > to be fast. > > It would be quite easy to make both cases FIFO and O(1). No synchronization > changes, just change a few lines of code while holding the same locks that are already > being held. > > If nobody objects, I think I will try this. > > >> > >> Mika's high-volume thread tester continuously churns out > >> "Thread appears starved or deadlocked" messages by the hundreds. > >> > >>> Sent from my iPad > >>> > >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: > >>>> > >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I > >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those > >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really > >>>> what we want? > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 30 17:04:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 30 Jun 2015 10:04:12 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: References: <559035DB.3040101@lcwb.coop>, , <55914AE0.8080407@lcwb.coop>, <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu>, <5591E061.8090400@lcwb.coop> Message-ID: <5592AFEC.70906@lcwb.coop> On 06/30/2015 01:37 AM, Jay K wrote: > 1) This sounds good. > 2) Can you please apply your analysis to the Win32 version also? > It looks to me like these matters are entirely delegated to windows-supplied library routines EnterCriticalSection, LeaveCriticalSection, SetEvent, and WaitForMultipleObjects. > > Tangentially, I've said both of these before: > > > 3) I'd really like to layer the pthread and Win32 > versions over a common interface. > > > 4) Possibly make it much thinner, if Modula-3 semantics > are close enough to pthread/win32+cv. > > > - Jay > > > > > Date: Mon, 29 Jun 2015 19:18:41 -0500 > > From: rodney_bates at lcwb.coop > > To: hosking at purdue.edu; mika at async.caltech.edu; m3devel at elegosoft.com > > Subject: Re: [M3devel] pthread scheduling fairness of Signal > > > > > > > > On 06/29/2015 08:53 AM, Antony Hosking wrote: > > > > > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: > > >> > > >> > > >> > > >> On 06/28/2015 01:16 PM, Antony Hosking wrote: > > >>> It mirrors the user-level threads information as far as I recall. > > >>> > > >> > > >> Do you think that was intentional in the original user-level implementation? > > >> It seems like an open invitation to starvation. And it seems even more > > >> important for condition variables than mutexen. (hush, spell checker!) > > > > > > Perhaps take a look at the original user-level implementation to confirm. > > > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > > > > > > > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening > > Mutex waiters and LIFO for Condition waiters. > > > > Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. > > (n=# of waiting threads.) I think n will most often be quite small, but occasionally, > > maybe not, and if/when high thread counts do occur, it would be a good time for things > > to be fast. > > > > It would be quite easy to make both cases FIFO and O(1). No synchronization > > changes, just change a few lines of code while holding the same locks that are already > > being held. > > > > If nobody objects, I think I will try this. > > > > >> > > >> Mika's high-volume thread tester continuously churns out > > >> "Thread appears starved or deadlocked" messages by the hundreds. > > >> > > >>> Sent from my iPad > > >>> > > >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: > > >>>> > > >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I > > >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those > > >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really > > >>>> what we want? > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> > > >> > > >> -- > > >> Rodney Bates > > >> rodney.m.bates at acm.org > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 00:15:26 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:15:26 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B4BE9.1000902@marino.st> References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. - Jay On May 31, 2015, at 10:59 AM, John Marino wrote: > I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > Should I put it back? From the last response I'm guessing that wasn't > the problem. > > John > > > On 5/31/2015 19:09, Jay K wrote: >> Oh probably the problem is using the backend premature actually. >> Darn, I can't help right now. >> >> - Jay >> >> >> >> ------------------------------------------------------------------------ >> From: jay.krell at cornell.edu >> To: adacore at marino.st; m3devel at elegosoft.com >> Date: Sun, 31 May 2015 16:45:42 +0000 >> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >> (0x100), expected 0x110 >> >> Hm. Strange. I need to read more closely. I see it skipping shipping, >> but my read was it didn't try that. >> So.. with INSTALL_CM3_IN_BIN=yes? >> I need to try this all, and "fix" it to set that. The whole thing in >> m3cc/src/m3makefile is relatively >> new vs. when I was actively using all this. :( >> >> - Jay >> >> >>> Date: Sun, 31 May 2015 16:57:02 +0200 >>> From: adacore at marino.st >>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >> (0x100), expected 0x110 >>> >>> On 5/31/2015 11:53, Jay K wrote: >>>> John, have you tried make-dist.py? >>>> i.e. >> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>> >>>> While it might not be exactly what you need, it should demonstrate the >>>> elements. >>>> >>>> I will try to try this all soon, really, I hope so. >>>> >>>> It starts with a working compiler, does no in-place updates, build into >>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>> It does "min" and "all". >>>> >>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>> >>>> - Jay >>> >>> Hi Jay, >>> I'm getting the same error as always using make-dist script: >>> http://leaf.dragonflybsd.org/~marino/m3d.log >>> >>> Thanks, >>> John From jay.krell at cornell.edu Mon Jun 1 00:47:42 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:47:42 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> Ok,caveat that I haven't tried this, could be more waste of your time,but in scripts/pkginfo.txt try moving the m3cc line down to just after cm3. This way, if the config is looking all around for it, as was mentioned and as we should avoid, it won't find it prematurely. I was never quite sure what order to put this entry, as it has in a sense no edges to or from it in the build dependency graph. But there is the sensitivity as to when to use it, so when to ship it. Shipping after cm3 should avoid using it prematurely. - Jay On May 31, 2015, at 3:15 PM, Jay wrote: > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > - Jay > > On May 31, 2015, at 10:59 AM, John Marino wrote: > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >> Should I put it back? From the last response I'm guessing that wasn't >> the problem. >> >> John >> >> >> On 5/31/2015 19:09, Jay K wrote: >>> Oh probably the problem is using the backend premature actually. >>> Darn, I can't help right now. >>> >>> - Jay >>> >>> >>> >>> ------------------------------------------------------------------------ >>> From: jay.krell at cornell.edu >>> To: adacore at marino.st; m3devel at elegosoft.com >>> Date: Sun, 31 May 2015 16:45:42 +0000 >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>> >>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>> but my read was it didn't try that. >>> So.. with INSTALL_CM3_IN_BIN=yes? >>> I need to try this all, and "fix" it to set that. The whole thing in >>> m3cc/src/m3makefile is relatively >>> new vs. when I was actively using all this. :( >>> >>> - Jay >>> >>> >>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>> From: adacore at marino.st >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>>> >>>> On 5/31/2015 11:53, Jay K wrote: >>>>> John, have you tried make-dist.py? >>>>> i.e. >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>> >>>>> While it might not be exactly what you need, it should demonstrate the >>>>> elements. >>>>> >>>>> I will try to try this all soon, really, I hope so. >>>>> >>>>> It starts with a working compiler, does no in-place updates, build into >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>> It does "min" and "all". >>>>> >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>> >>>>> - Jay >>>> >>>> Hi Jay, >>>> I'm getting the same error as always using make-dist script: >>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>> >>>> Thanks, >>>> John From jay.krell at cornell.edu Mon Jun 1 00:51:25 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 31 May 2015 15:51:25 -0700 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> References: <556700B0.8060500@marino.st> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> Message-ID: <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> Ps. Update pkginfo.txt and then try make-dist.py - Jay On May 31, 2015, at 3:47 PM, Jay wrote: > Ok,caveat that I haven't tried this, could be more waste of your time,but in scripts/pkginfo.txt try moving the m3cc line down to just after cm3. This way, if the config is looking all around for it, as was mentioned and as we should avoid, it won't find it prematurely. > > > I was never quite sure what order to put this entry, as it has in a sense no edges to or from it in the build dependency graph. But there is the sensitivity as to when to use it, so when to ship it. Shipping after cm3 should avoid using it prematurely. > > - Jay > > On May 31, 2015, at 3:15 PM, Jay wrote: > >> I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. >> >> - Jay >> >> On May 31, 2015, at 10:59 AM, John Marino wrote: >> >>> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >>> Should I put it back? From the last response I'm guessing that wasn't >>> the problem. >>> >>> John >>> >>> >>> On 5/31/2015 19:09, Jay K wrote: >>>> Oh probably the problem is using the backend premature actually. >>>> Darn, I can't help right now. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> From: jay.krell at cornell.edu >>>> To: adacore at marino.st; m3devel at elegosoft.com >>>> Date: Sun, 31 May 2015 16:45:42 +0000 >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>>> (0x100), expected 0x110 >>>> >>>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>>> but my read was it didn't try that. >>>> So.. with INSTALL_CM3_IN_BIN=yes? >>>> I need to try this all, and "fix" it to set that. The whole thing in >>>> m3cc/src/m3makefile is relatively >>>> new vs. when I was actively using all this. :( >>>> >>>> - Jay >>>> >>>> >>>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>>> From: adacore at marino.st >>>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>>> (0x100), expected 0x110 >>>>> >>>>> On 5/31/2015 11:53, Jay K wrote: >>>>>> John, have you tried make-dist.py? >>>>>> i.e. >>>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>>> >>>>>> While it might not be exactly what you need, it should demonstrate the >>>>>> elements. >>>>>> >>>>>> I will try to try this all soon, really, I hope so. >>>>>> >>>>>> It starts with a working compiler, does no in-place updates, build into >>>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>>> It does "min" and "all". >>>>>> >>>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>>> >>>>>> - Jay >>>>> >>>>> Hi Jay, >>>>> I'm getting the same error as always using make-dist script: >>>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>>> >>>>> Thanks, >>>>> John From hendrik at topoi.pooq.com Mon Jun 1 02:29:44 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 31 May 2015 20:29:44 -0400 Subject: [M3devel] User interfaces for Modula 3. Message-ID: <20150601002944.GA10751@topoi.pooq.com> What user interface libraries are available for Modula 3? I know there's Trestle. But is there something like GTK or QT or somethng I can use like cairo to draw really pretty text? Anything that supports lots of Unicode? I don't mind havein to do my own character placement based on font metrics of some sort; not now, bit later, I may have some really weird layout constraints. In case you're wodering about the application: I'm doing preliminary planning for something like a text editor with several windows, one with the raw text, and another with a continuous (as continuous as I have cpu cycles for) display of the results of rather complex calculations on that text (such as proof checking or described graphics). Modula 3 isn't the only candidate for a programming language, just the leading candidate for historical and bootstrapping reasons. The others at the moment are OCAML and some kind of statically typed Scheme. And of course, I may decide that theo whole projecct is infeasible. -- hendrik From rodney_bates at lcwb.coop Mon Jun 1 05:28:13 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 31 May 2015 22:28:13 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> Message-ID: <556BD14D.5050507@lcwb.coop> I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes to fail regardless of the order of building the two. OTOH, if cm3 were built and shipped, then m3cc came immediately after, it should work, because m3cc contains no Modula-3 code, and the new cm3 would not, when building it, run cm3cg. After that, there would be a consistent set. for further compiles. But any package containing M3 code built between cm3 and m3cc would fail. On 05/31/2015 05:15 PM, Jay wrote: > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > - Jay > > On May 31, 2015, at 10:59 AM, John Marino wrote: > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. >> Should I put it back? From the last response I'm guessing that wasn't >> the problem. >> >> John >> >> >> On 5/31/2015 19:09, Jay K wrote: >>> Oh probably the problem is using the backend premature actually. >>> Darn, I can't help right now. >>> >>> - Jay >>> >>> >>> >>> ------------------------------------------------------------------------ >>> From: jay.krell at cornell.edu >>> To: adacore at marino.st; m3devel at elegosoft.com >>> Date: Sun, 31 May 2015 16:45:42 +0000 >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>> >>> Hm. Strange. I need to read more closely. I see it skipping shipping, >>> but my read was it didn't try that. >>> So.. with INSTALL_CM3_IN_BIN=yes? >>> I need to try this all, and "fix" it to set that. The whole thing in >>> m3cc/src/m3makefile is relatively >>> new vs. when I was actively using all this. :( >>> >>> - Jay >>> >>> >>>> Date: Sun, 31 May 2015 16:57:02 +0200 >>>> From: adacore at marino.st >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp >>> (0x100), expected 0x110 >>>> >>>> On 5/31/2015 11:53, Jay K wrote: >>>>> John, have you tried make-dist.py? >>>>> i.e. >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py >>>>> >>>>> While it might not be exactly what you need, it should demonstrate the >>>>> elements. >>>>> >>>>> I will try to try this all soon, really, I hope so. >>>>> >>>>> It starts with a working compiler, does no in-place updates, build into >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. >>>>> It does "min" and "all". >>>>> >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. >>>>> >>>>> - Jay >>>> >>>> Hi Jay, >>>> I'm getting the same error as always using make-dist script: >>>> http://leaf.dragonflybsd.org/~marino/m3d.log >>>> >>>> Thanks, >>>> John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 05:32:29 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 31 May 2015 22:32:29 -0500 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <20150601002944.GA10751@topoi.pooq.com> References: <20150601002944.GA10751@topoi.pooq.com> Message-ID: <556BD24D.7010902@lcwb.coop> There is a QT binding in cm3/m3-ui/qt. I have no experience with it. On 05/31/2015 07:29 PM, Hendrik Boom wrote: > What user interface libraries are available for Modula 3? > > I know there's Trestle. > > But is there something like GTK or QT or somethng I can use like cairo > to draw really pretty text? Anything that supports lots of Unicode? > I don't mind havein to do my own character placement based on font > metrics of some sort; not now, bit later, I may have some really > weird layout constraints. > > In case you're wodering about the application: > > I'm doing preliminary planning for something like a text editor with > several windows, one with the raw text, and another with a continuous > (as continuous as I have cpu cycles for) display of the results of rather > complex calculations on that text (such as proof checking or > described graphics). > > Modula 3 isn't the only candidate for a programming language, just the > leading candidate for historical and bootstrapping reasons. The others > at the moment are OCAML and some kind of statically typed Scheme. > > And of course, I may decide that theo whole projecct is infeasible. > > -- hendrik > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 08:40:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:40:38 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556BD14D.5050507@lcwb.coop> References: <556700B0.8060500@marino.st> <5567DA8C.3050802@lcwb.coop>,<556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , <556BD14D.5050507@lcwb.coop> Message-ID: Yes, exactly, that's what I was trying to say, in my later/last mail. Yes the compability is definitely two-way. There is a one to one mapping back and forth. cm3 goes with cm3cg; cm3cg goes with cm3. Neither is compatible with older nor necessarily newer versions. They would be statically linked together if not for the GPL. When I first placed m3cc in pkginfo.txt years ago, I was only thinking of build order and, like, "override". m3cc imports nothing and nothing imports m3cc. In the Modula-3 module/library/interface sense. It can be built when you have nothing. So I figured put it nearly first. Only now today did I consider the shipping order. It should be shipped adjacent to cm3, either right before or right after. If you split build and ship, then it can still be built early, or at almost any time. But we have one ordering for build and ship, so let the shipping order decide. I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. I still think it broke upgrade.py. I can workaround. But I'd like to remove the environment variable check. There are many order dependencies in build and ship. This is just one. - Jay > Date: Sun, 31 May 2015 22:28:13 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; adacore at marino.st > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > to fail regardless of the order of building the two. > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > it should work, because m3cc contains no Modula-3 code, and the new > cm3 would not, when building it, run cm3cg. After that, there would > be a consistent set. for further compiles. But any package containing > M3 code built between cm3 and m3cc would fail. > > On 05/31/2015 05:15 PM, Jay wrote: > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > - Jay > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > >> Should I put it back? From the last response I'm guessing that wasn't > >> the problem. > >> > >> John > >> > >> > >> On 5/31/2015 19:09, Jay K wrote: > >>> Oh probably the problem is using the backend premature actually. > >>> Darn, I can't help right now. > >>> > >>> - Jay > >>> > >>> > >>> > >>> ------------------------------------------------------------------------ > >>> From: jay.krell at cornell.edu > >>> To: adacore at marino.st; m3devel at elegosoft.com > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >>> (0x100), expected 0x110 > >>> > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > >>> but my read was it didn't try that. > >>> So.. with INSTALL_CM3_IN_BIN=yes? > >>> I need to try this all, and "fix" it to set that. The whole thing in > >>> m3cc/src/m3makefile is relatively > >>> new vs. when I was actively using all this. :( > >>> > >>> - Jay > >>> > >>> > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > >>>> From: adacore at marino.st > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >>> (0x100), expected 0x110 > >>>> > >>>> On 5/31/2015 11:53, Jay K wrote: > >>>>> John, have you tried make-dist.py? > >>>>> i.e. > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > >>>>> > >>>>> While it might not be exactly what you need, it should demonstrate the > >>>>> elements. > >>>>> > >>>>> I will try to try this all soon, really, I hope so. > >>>>> > >>>>> It starts with a working compiler, does no in-place updates, build into > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > >>>>> It does "min" and "all". > >>>>> > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > >>>>> > >>>>> - Jay > >>>> > >>>> Hi Jay, > >>>> I'm getting the same error as always using make-dist script: > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > >>>> > >>>> Thanks, > >>>> John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 08:54:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:54:29 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B5356.40809@lcwb.coop> References: <556700B0.8060500@marino.st>, <5567DA8C.3050802@lcwb.coop>,,<556817A1.2090206@marino.st>, ,,<20150529105451.e9c9b9c2a5578a28bc654f97@elego.de>, ,,<55682D1A.2030504@marino.st>, ,,<20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de>, ,,<55683689.2090709@marino.st>, ,,<20150529122732.1396eb079ff9b404bcb21e40@elego.de>, ,,<5568421D.3050209@marino.st>, ,,<20150529155833.8454664b88391d93708e8f85@elego.de>, ,,<55687266.3090400@marino.st>, ,,<20150529162028.388911007a614006049beb42@elego.de>, ,,<20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, ,,<556885FD.6010800@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, ,,<55689065.1040207@marino.st>, ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, ,,, , , <556B213E.8030108@marino.st>, , , <556B4BE9.1000902@marino .st>,<556B5356.40809@lcwb.coop> Message-ID: It only happens if you ship. Shipping in the wrong order can always break things. You cannot ship in arbitrary order and incorrect order can cause arbitrary problems. That doesn't mean we should never ship and always have a separate copy mechanism. The knowledge of what to ship is meant to be in the m3makefiles. Not also encoded elsewhere. The environment variable check really should be removed. Perhaps perhaps perh aps shipping order can be enforced? For things other than m3cc, this seems possible. Immediate matching dependencies should be present in the destination. Using the information that is already present in the system as to immediate and matching, in terms of Modula-3 imports/interfaces. m3cc is unique in not fitting in that however. As well, the dependency between m3cc/cm3 -- what order would you say it is? Circular but the tie can be broken arbitrarily? Introduce a command that can ship multiple packages at once? I kind of want that anyway, but in this case you'd arrive back at the environment variable, as long as cm3 has it..though cm3 only needs it for NT and hypothetical other targets like AIX/OS2/DOS/DJGPP. However -- how about this suggestion -- make the environment variable check for relative native target? Only skip cm3 shipping when we know it doesn't work, I..e NT/Cygwin/Mingwin? I like this. It is a special case either way, but at least we can demonstrate a cleaner system on most systems.. - Jay > Date: Sun, 31 May 2015 13:30:46 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; adacore at marino.st > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 12:59 PM, John Marino wrote: > > I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > Should I put it back? From the last response I'm guessing that wasn't > > the problem. > > > > John > > > > It apparently wasn't the only problem, but it definitely was a problem. > Don't put it back. I causes the new cm3cg executable to be copied > to your bin directory immediately after it has been built. It is > incompatible with the old cm3 executable, and the next attempted > build (m3core, as I remember) runs the old cm3, then the new cm3cg, > which correctly recognizes that its input is in an old version format > and refuses, giving the error message. > > > > > On 5/31/2015 19:09, Jay K wrote: > >> Oh probably the problem is using the backend premature actually. > >> Darn, I can't help right now. > >> > >> - Jay > >> > >> > >> > >> ------------------------------------------------------------------------ > >> From: jay.krell at cornell.edu > >> To: adacore at marino.st; m3devel at elegosoft.com > >> Date: Sun, 31 May 2015 16:45:42 +0000 > >> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >> (0x100), expected 0x110 > >> > >> Hm. Strange. I need to read more closely. I see it skipping shipping, > >> but my read was it didn't try that. > >> So.. with INSTALL_CM3_IN_BIN=yes? > >> I need to try this all, and "fix" it to set that. The whole thing in > >> m3cc/src/m3makefile is relatively > >> new vs. when I was actively using all this. :( > >> > >> - Jay > >> > >> > >>> Date: Sun, 31 May 2015 16:57:02 +0200 > >>> From: adacore at marino.st > >>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > >> (0x100), expected 0x110 > >>> > >>> On 5/31/2015 11:53, Jay K wrote: > >>>> John, have you tried make-dist.py? > >>>> i.e. > >> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > >>>> > >>>> While it might not be exactly what you need, it should demonstrate the > >>>> elements. > >>>> > >>>> I will try to try this all soon, really, I hope so. > >>>> > >>>> It starts with a working compiler, does no in-place updates, build into > >>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > >>>> It does "min" and "all". > >>>> > >>>> If you set the STAGE environment variable, it uses that instead of /tmp. > >>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > >>>> > >>>> - Jay > >>>> > >>> > >>> Hi Jay, > >>> I'm getting the same error as always using make-dist script: > >>> http://leaf.dragonflybsd.org/~marino/m3d.log > >>> > >>> Thanks, > >>> John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 08:48:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 06:48:12 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B5D6A.3030403@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B5D6A.3030403@lcwb.coop> Message-ID: > was looking several alternative places for an executable cm3cg I agree it did/does do that. I did put that in. I thought it was a good idea at the time. I realize now it is a mistake. If that behavior is still present, it will be advisable also to first update one's config to stop doing that. Basically, "ship" is a very carefully ordered operation, the shipping of multiple packages. If you have no bugs, and you ship in the correct order, then you don't need DESTDIR. IF you have no bugs, but you ship in the incorrect order, you cause a bad mixing of things. If you use "DESTDIR" then you can ship safely in any order and do some switch at the end, such as by reseting PATH to point to the new set of files. If you have circular dependencies, there might not be a workable order. I agree this could be the case. make-dist.py is written to avoid but might be slightly wrong. I suspect I only ever ran make-dist after upgrade. I have started setting up VMs..though NetBSD, FreeBSD, OpenBSD all fail under VirtualBox if Hyper-V is enabled...but at least Debian is making progress. So hopefully I can test and fix this properly soon. And then I'll back to using the C backend anyway. :) - Jay > Date: Sun, 31 May 2015 14:13:46 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > The error message you keep getting pretty certainly means it is running a > new cm3cg with an old cm3. And it consistently happens immediately after > cm3cg is rebuilt. > > The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > your failed builds, and I don't think that package would ever run cm3cg, as > there is no Modula-3 code in it. Which means there is no proof that the > version of cm3cg that would have been used before it was rebuilt is the old > one. (Could it be a new one left over from a previous incomplete build?) > > One experiment that would prove this would be, in the state your are > starting from before trying a full build, try any means of running > cm3 on a package that contains Modula-3 code, to see if it avoids the > recurring problem. > > But I have another theory. I don't know what this ports bootstrap > is, but there was a time when things where set up so that cm3, instead > of looking in just the same directory as its own executable was found, > was looking several alternative places for an executable cm3cg. This > caused the same problem to occur in a different way. Even when the > newly rebuilt cm3cg executable was not copied to the bin directory > right away, its mere existence where it was built was causing it to > be found the next time cm3 started up, leading to the same failure. > > I don't remember how this was fixed, but either Jay or I did something > about it. Perhaps the ports bootstrap was made during a time when this > behavior was happening. > > On 05/31/2015 02:41 AM, John Marino wrote: > > On 5/29/2015 20:43, John Marino wrote: > >> On 5/29/2015 20:15, Olaf Wagner wrote: > >>> > >>> Well, yes, I understand that. I would have tried your exact setup, > >>> but I have no system available to test that on. > >>> > >>> At least I have validated that based on the origianl 5.8.6 installation > >>> archive for AMD64_FREEBSD you can build the new compiler from the current > >>> sources with a simple call of the upgrade.sh script. which I still doubted > >>> yesterday. > >> > >> > >> The card I still have left to play is to extract the bootstrap, let it > >> overwrite itself per Rodney's technique and then build the real compiler > >> (dumping the whole "intermediate" area). > > > > FWIW, this did not work either. Rodney's technique doesn't seem to work > > with the ports bootstrap. I don't know why it would work with provided > > 5.8.6 but not rebuilt one. As far as I can tell, I did what he said > > would always work. > > > > I'm at a loss as to where to go from here. It's starting to look like I > > have to throw away the ports 5.8.6 bootstrap and start over somehow. I > > don't think it should be this hard. :( > > > > John > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 1 09:05:31 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:05:31 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B743E.7030504@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B5D6A.3030403@lcwb.coop>, <556B6075.6010703@marino.st>, <556B743E.7030504@lcwb.coop> Message-ID: > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > 274: foreach root in [ m3cc, bin ] > to: > 274: foreach root in [ bin ] > Before building the Modula-3 head. Yes, agreed. I still want the environment variable check removed, but this is another problem, that I caused, and I did fix long ago, but is causing grief still, sorry about that. Here, I fixed it in 2010: https://github.com/modula3/cm3/commit/65551f30a6c3ed2cd9740394a9082ffab1b07faa INSTALL_ROOT/bin/cm3cg for native builds ROOT/m3-sys/m3cc/HOST/TARGET/cm3cg for cross builds "Later" (never?) we can think about "installed" cross builds. In particular, when I make m3cg interface changes, I get burned by reaching in for ROOT/m3-sys/m3cc/x/cm3cg. The code is a lot smaller now and more correct. Sorry about that. I was being ambitious in trying to make things work, and it turned out to be quite incorrect once things are better understood. - Jay > Date: Sun, 31 May 2015 15:51:10 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 02:26 PM, John Marino wrote: > > On 5/31/2015 21:13, Rodney M. Bates wrote: > >> The error message you keep getting pretty certainly means it is running a > >> new cm3cg with an old cm3. And it consistently happens immediately after > >> cm3cg is rebuilt. > >> > >> The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > >> your failed builds, and I don't think that package would ever run cm3cg, as > >> there is no Modula-3 code in it. Which means there is no proof that the > >> version of cm3cg that would have been used before it was rebuilt is the old > >> one. (Could it be a new one left over from a previous incomplete build?) > > > > That's not possible. The bootstrap is packaged in a compressed tarball > > and extracted for the sole purpose of building M3. It would have not > > leftovers in it. > > > > It comes with cm3, cm3cg, m3bundle, and mklib in the bin directory. > > > > the contents of the bootstrap M3 cm3.cfg are: > > INSTALL_ROOT = path() & "/.." > > include (path() & "/../etc/modula3/AMD64_FREEBSD") > > > > > >> One experiment that would prove this would be, in the state your are > >> starting from before trying a full build, try any means of running > >> cm3 on a package that contains Modula-3 code, to see if it avoids the > >> recurring problem. > > > > I think this is N/A. The bootstrap compiler is extracted to a directory > > that is not on the path (and only exists right before trying to build > > modula3 port) > > > > > >> But I have another theory. I don't know what this ports bootstrap > >> is, but there was a time when things where set up so that cm3, instead > >> of looking in just the same directory as its own executable was found, > >> was looking several alternative places for an executable cm3cg. This > > > > It's 5.8.6 and the cm3.cfg is listed above. You can download it > > yourself here: > > http://downloads.dragonlace.net/m3/m3-bootstrap.AMD64.FREEBSD.92.tar.bz2 > > > > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > > 274: foreach root in [ m3cc, bin ] > > to: > > 274: foreach root in [ bin ] > > Before building the Modula-3 head. > > > > >> caused the same problem to occur in a different way. Even when the > >> newly rebuilt cm3cg executable was not copied to the bin directory > >> right away, its mere existence where it was built was causing it to > >> be found the next time cm3 started up, leading to the same failure. > >> > >> I don't remember how this was fixed, but either Jay or I did something > >> about it. Perhaps the ports bootstrap was made during a time when this > >> behavior was happening. > > > > Probably not as I am 99% sure the bootstrap was built from the release > > source. > > > > Thanks, > > John > > > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 09:07:25 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 09:07:25 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> References: <556700B0.8060500@marino.st> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> Message-ID: <556C04AD.5000509@marino.st> On 6/1/2015 00:51, Jay wrote: > Ps. Update pkginfo.txt and then try make-dist.py > Okay, I applied this patch: http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt and the same thing happened, only later: http://leaf.dragonflybsd.org/~marino/m3e.log John From jay.krell at cornell.edu Mon Jun 1 09:05:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:05:05 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556B743E.7030504@lcwb.coop> References: <556700B0.8060500@marino.st> <55672DC1.7080509@lcwb.coop> <55673AA4.8050100@marino.st> <20150528190643.faf81897fc5698a06bf37838@elegosoft.com> <55679387.8040701@lcwb.coop> <5567DA8C.3050802@lcwb.coop> <556817A1.2090206@marino.st> <20150529105451.e9c9b9c2a5578a28bc654f97@elego.de> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B5D6A.3030403@lcwb.coop>, <556B6075.6010703@marino.st>, <556B743E.7030504@lcwb.coop> Message-ID: > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > 274: foreach root in [ m3cc, bin ] > to: > 274: foreach root in [ bin ] > Before building the Modula-3 head. Yes, agreed. I still want the environment variable check removed, but this is another problem, that I caused, and I did fix long ago, but is causing grief still, sorry about that. Here, I fixed it in 2010: https://github.com/modula3/cm3/commit/65551f30a6c3ed2cd9740394a9082ffab1b07faa INSTALL_ROOT/bin/cm3cg for native builds ROOT/m3-sys/m3cc/HOST/TARGET/cm3cg for cross builds "Later" (never?) we can think about "installed" cross builds. In particular, when I make m3cg interface changes, I get burned by reaching in for ROOT/m3-sys/m3cc/x/cm3cg. The code is a lot smaller now and more correct. Sorry about that. I was being ambitious in trying to make things work, and it turned out to be quite incorrect once things are better understood. - Jay > Date: Sun, 31 May 2015 15:51:10 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 05/31/2015 02:26 PM, John Marino wrote: > > On 5/31/2015 21:13, Rodney M. Bates wrote: > >> The error message you keep getting pretty certainly means it is running a > >> new cm3cg with an old cm3. And it consistently happens immediately after > >> cm3cg is rebuilt. > >> > >> The build of cm3cg (package m3cc) appears to be the first use of cm3 in all > >> your failed builds, and I don't think that package would ever run cm3cg, as > >> there is no Modula-3 code in it. Which means there is no proof that the > >> version of cm3cg that would have been used before it was rebuilt is the old > >> one. (Could it be a new one left over from a previous incomplete build?) > > > > That's not possible. The bootstrap is packaged in a compressed tarball > > and extracted for the sole purpose of building M3. It would have not > > leftovers in it. > > > > It comes with cm3, cm3cg, m3bundle, and mklib in the bin directory. > > > > the contents of the bootstrap M3 cm3.cfg are: > > INSTALL_ROOT = path() & "/.." > > include (path() & "/../etc/modula3/AMD64_FREEBSD") > > > > > >> One experiment that would prove this would be, in the state your are > >> starting from before trying a full build, try any means of running > >> cm3 on a package that contains Modula-3 code, to see if it avoids the > >> recurring problem. > > > > I think this is N/A. The bootstrap compiler is extracted to a directory > > that is not on the path (and only exists right before trying to build > > modula3 port) > > > > > >> But I have another theory. I don't know what this ports bootstrap > >> is, but there was a time when things where set up so that cm3, instead > >> of looking in just the same directory as its own executable was found, > >> was looking several alternative places for an executable cm3cg. This > > > > It's 5.8.6 and the cm3.cfg is listed above. You can download it > > yourself here: > > http://downloads.dragonlace.net/m3/m3-bootstrap.AMD64.FREEBSD.92.tar.bz2 > > > > OK, the fix I mentioned is in the 5.8 release branch on github, but it is > not in the above bootstrap. In file bootstrap/etc/modula3/cm3cfg.common, > line 274 needs to change from: > > 274: foreach root in [ m3cc, bin ] > > to: > > 274: foreach root in [ bin ] > > Before building the Modula-3 head. > > > > >> caused the same problem to occur in a different way. Even when the > >> newly rebuilt cm3cg executable was not copied to the bin directory > >> right away, its mere existence where it was built was causing it to > >> be found the next time cm3 started up, leading to the same failure. > >> > >> I don't remember how this was fixed, but either Jay or I did something > >> about it. Perhaps the ports bootstrap was made during a time when this > >> behavior was happening. > > > > Probably not as I am 99% sure the bootstrap was built from the release > > source. > > > > Thanks, > > John > > > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Mon Jun 1 09:11:22 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 1 Jun 2015 09:11:22 +0200 (CEST) Subject: [M3devel] duplicate subscription Message-ID: It seems that I am subscribed twice to this list. How can I find out the according e-mail addresses? From adacore at marino.st Mon Jun 1 09:19:01 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 09:19:01 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C04AD.5000509@marino.st> References: <556700B0.8060500@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> Message-ID: <556C0765.5050607@marino.st> On 6/1/2015 09:07, John Marino wrote: > On 6/1/2015 00:51, Jay wrote: >> Ps. Update pkginfo.txt and then try make-dist.py >> > > Okay, I applied this patch: > http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt > > and the same thing happened, only later: > http://leaf.dragonflybsd.org/~marino/m3e.log > Well, not the "same" thing. ${STAGE}/compiler_with_self/bin seems to be installed. I didn't change bootstrap/etc/modula3/cm3cfg.common though, it sounds like I shouldn't after all. Should I? John From jay.krell at cornell.edu Mon Jun 1 09:35:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 07:35:19 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C0765.5050607@marino.st> References: <556700B0.8060500@marino.st>, <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st> Message-ID: Given that you seem to be very quick and very patient, then yes, try that. I am trying to get back up to speed in all this, but you are way ahead of me. The actual fix in 2010 was more than Rodney described. It was removing a bunch of code. The theory is that we are still reaching around and finding the wrong cm3cg somewhere, instead of going directly to the matching one in the one place it belongs (ignoring cross build matters). It isn't entirely clear though, like where would it find the wrong one. I understand how it used to go bad . see: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common I hope to get catch up soon.. :( on being away.. - Jay > Date: Mon, 1 Jun 2015 09:19:01 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 09:07, John Marino wrote: > > On 6/1/2015 00:51, Jay wrote: > >> Ps. Update pkginfo.txt and then try make-dist.py > >> > > > > Okay, I applied this patch: > > http://leaf.dragonflybsd.org/~marino/patch-scripts_pkginfo.txt > > > > and the same thing happened, only later: > > http://leaf.dragonflybsd.org/~marino/m3e.log > > > > > Well, not the "same" thing. > ${STAGE}/compiler_with_self/bin seems to be installed. > > I didn't change bootstrap/etc/modula3/cm3cfg.common though, it sounds > like I shouldn't after all. Should I? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Jun 1 10:02:13 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 1 Jun 2015 10:02:13 +0200 Subject: [M3devel] duplicate subscription In-Reply-To: References: Message-ID: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> On Mon, 1 Jun 2015 09:11:22 +0200 (CEST) Henning Thielemann wrote: > > It seems that I am subscribed twice to this list. How can I find out the > according e-mail addresses? Mailing list information can be found at https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel (or /m3commit, /m3announce). Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From adacore at marino.st Mon Jun 1 10:04:40 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:04:40 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st> Message-ID: <556C1218.3000103@marino.st> On 6/1/2015 09:35, Jay K wrote: > Given that you seem to be very quick and very patient, then yes, try that. > I am trying to get back up to speed in all this, but you are way ahead > of me. After this morning, I'll have to put this away until the weekend I think. However, changing that file in addition almost worked: http://leaf.dragonflybsd.org/~marino/m3f.log Failed at the end of log: "/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib/src/../../quake/cvsup.quake", line 117: quake runtime error: undefined variable: IsDarwin I'm guessing a simple patch to cvsup.quake would be sufficient, although it would me the error is current. John From wagner at elegosoft.com Mon Jun 1 10:13:09 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 1 Jun 2015 10:13:09 +0200 Subject: [M3devel] cm3/m3quake: setting environment variables for exec In-Reply-To: <556B49DD.20909@elstel.org> References: <556B49DD.20909@elstel.org> Message-ID: <20150601101309.2ac490ee3305b476a29be2ec@elegosoft.com> On Sun, 31 May 2015 19:50:21 +0200 Elmar Stellnberger wrote: > How can I set environment variables for exec with cm3? > Using /usr/bin/env would be inherently non-portable. > With SRC M3 I remember this was the 4th parameter or so. > However cm3 exec seems to do something very different. I don't know about that. The complete quake specification can be found at http://www.opencm3.net/doc/help/cm3/quake.html including all built-in functions and procedures. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 1 10:15:35 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 08:15:35 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1218.3000103@marino.st> References: <556700B0.8060500@marino.st>, <20150529122732.1396eb079ff9b404bcb21e40@elego.de>, <5568421D.3050209@marino.st>, <20150529155833.8454664b88391d93708e8f85@elego.de>, <55687266.3090400@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, <556885FD.6010800@marino.st>, <20150529180223.d467621b68ebf665da685b12@elego.de>, <55689065.1040207@marino.st>, <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B213E.8030108@marino.st>, , <556B4BE9.1000902@marino.st>, , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , <556C1218.3000103@marino.st> Message-ID: This is very good progress. Thank you for your speed and patience. And the error is kind of minor and uninteresting. Yes, I understand, it is fatal. Everything should just work to completion. See IsDarwin here: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common cvsup is fiddling around how to use zlib. See here: https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile and surely cvsup isn't interesting any more? Who is using cvs? You can delete m3-tools/cvsup. or change the scripts to skip it or update cm3cfg.common to be more up to date. which brings me to...seems like maybe an incorrect mixing of bootstrap config and latest config? Which begs the question though to which parts of config are local-specific and shouldn't be updated/replaced, and which parts are bound to cm3 and should be replaced. Which really suggest a slight refactoring of the config files. They are nicely factored, but not quite along these lines. Arguably whatever is bound to cm3 should be written in Modula-3 and statically linked, not in these free floating text files. But quake is convenient, and fast enough seemingly... - Jay > Date: Mon, 1 Jun 2015 10:04:40 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 09:35, Jay K wrote: > > Given that you seem to be very quick and very patient, then yes, try that. > > I am trying to get back up to speed in all this, but you are way ahead > > of me. > > After this morning, I'll have to put this away until the weekend I think. > > However, changing that file in addition almost worked: > http://leaf.dragonflybsd.org/~marino/m3f.log > > Failed at the end of log: > "/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib/src/../../quake/cvsup.quake", > line 117: quake runtime error: undefined variable: IsDarwin > > > I'm guessing a simple patch to cvsup.quake would be sufficient, although > it would me the error is current. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 10:25:11 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:25:11 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <5568421D.3050209@marino.st>, <20150529155833.8454664b88391d93708e8f85@elego.de>, <55687266.3090400@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, <556885FD.6010800@marino.st>, <20150529180223.d467621b68ebf665da685b12@elego.de>, <55689065.1040207@marino.st>, <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, <556B213E.8030108@marino.st>, , <556B4BE9.1000902@marino.st>, , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , <556C1218.3000103@marino.st> Message-ID: <556C16E7.505@marino.st> On 6/1/2015 10:15, Jay K wrote: > This is very good progress. > Thank you for your speed and patience. > > > And the error is kind of minor and uninteresting. Yes, I understand, it > is fatal. Everything should just work to completion. > > > See IsDarwin here: > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > cvsup is fiddling around how to use zlib. > See here: > https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile > > > and surely cvsup isn't interesting any more? Who is using cvs? NetBSD and OpenBSD for starters. > You can delete m3-tools/cvsup. > or change the scripts to skip it > or update cm3cfg.common to be more up to date. I'd like to keep it, it's the best source for cvsup. I thought make-dist.py was just going to bootstrap (e.g. minimal required). Is it actually building everything? I was intending to run do-cm3-min.sh buildship, do-cm3-std.sh buildship, do-cm3-caltech-parser.sh buildship after make-dist.py completed. John From dmuysers at hotmail.com Mon Jun 1 10:26:37 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Mon, 1 Jun 2015 10:26:37 +0200 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <20150601002944.GA10751@topoi.pooq.com> References: <20150601002944.GA10751@topoi.pooq.com> Message-ID: Yes, there is libiup. Interface is portable C. Remains to write an M3 interface file. And it is pure Unicode (UTF-8), as it should be everywhere -----Original Message----- From: Hendrik Boom Sent: Monday, June 01, 2015 2:29 AM To: m3devel Subject: [M3devel] User interfaces for Modula 3. What user interface libraries are available for Modula 3? I know there's Trestle. But is there something like GTK or QT or somethng I can use like cairo to draw really pretty text? Anything that supports lots of Unicode? I don't mind havein to do my own character placement based on font metrics of some sort; not now, bit later, I may have some really weird layout constraints. In case you're wodering about the application: I'm doing preliminary planning for something like a text editor with several windows, one with the raw text, and another with a continuous (as continuous as I have cpu cycles for) display of the results of rather complex calculations on that text (such as proof checking or described graphics). Modula 3 isn't the only candidate for a programming language, just the leading candidate for historical and bootstrapping reasons. The others at the moment are OCAML and some kind of statically typed Scheme. And of course, I may decide that theo whole projecct is infeasible. -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: wlEmoticon-smile[1].png Type: image/png Size: 1046 bytes Desc: not available URL: From adacore at marino.st Mon Jun 1 10:58:59 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 10:58:59 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <20150529162028.388911007a614006049beb42@elego.de>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> Message-ID: <556C1ED3.3000303@marino.st> On 6/1/2015 10:53, Jay K wrote: > make-dist makes everything. > It makes a "min" distribution and an "all" distribution. > Hopefully you can read it through it makes semi-reasonable sense. yes, I already figured this out. > This is all my doing -- which is to say, I quite like it, > but not necessarily anyone else. > min as I recall, is just the compiler: cm3, cm3cg, mklib, config, > m3core, libm3 > (mklib is only for NT386 target, but it is small and innocuous) > all is everything, including min > I don't like the confusion and purported complexity of breaking > things down into several medium sized packages, with dependencies. > I want more of a "one stop shop", granted, a big one, that > gives everyone stuff they won't use. > But I compromised and it makes to things. Right now, I'm assuming I can write an "install target" that copies from $STAGE/$DISTRIBUTION to where it needs to be installed in ports. If so, this approach is fine. Simpler even. > > cvs is pretty awful. > I'm still more productive in it than git, but I expect that will change. > But ok. cvsup can work. The error is for a trivial thing. I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and hopefully that will work. I don't know quake language. (I don't know why IsDarwin does't work because it's defined in the new cm3 distibution's cm3cfg.common.) Let's see how it goes. John From jay.krell at cornell.edu Mon Jun 1 10:53:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 08:53:00 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C16E7.505@marino.st> References: <556700B0.8060500@marino.st>, <5568421D.3050209@marino.st>, , <20150529155833.8454664b88391d93708e8f85@elego.de>, , <55687266.3090400@marino.st>, , <20150529162028.388911007a614006049beb42@elego.de>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> Message-ID: make-dist makes everything. It makes a "min" distribution and an "all" distribution. Hopefully you can read it through it makes semi-reasonable sense. This is all my doing -- which is to say, I quite like it, but not necessarily anyone else. min as I recall, is just the compiler: cm3, cm3cg, mklib, config, m3core, libm3 (mklib is only for NT386 target, but it is small and innocuous) all is everything, including min I don't like the confusion and purported complexity of breaking things down into several medium sized packages, with dependencies. I want more of a "one stop shop", granted, a big one, that gives everyone stuff they won't use. But I compromised and it makes to things. It makes roughly speaking .tar.gz files. Or .zips for some targets. Or compresed tar, depending on what compressor it can find, and bundles them into .deb files. But .deb files are little more than .tar.gz or .tar.bz2 or .tar.xz upgrade.sh also builds everything When I wrote it to upgrade.py I missed that significant aspect, and it only builds the compiler. Perhaps I should rename it to upgrade-min.py and then have upgrade-all.py. You can mimic upgrade.sh with ugprade.py && do-cm3-all.py realclean skipgcc && do-cm3-all.py buildship (skipgcc is just an optimization to avoid some duplicate work) I never use any of the .sh files. I found them to hard to understand and not portable enough. i.e. I'd rather install Python on Windows than Cygwin. I suspect Bourne shell is just not worth learning to program in. I had rewritten them in .cmd for Windows, but wanted one set of scripts. cvs is pretty awful. I'm still more productive in it than git, but I expect that will change. But ok. cvsup can work. The error is for a trivial thing. - Jay > Date: Mon, 1 Jun 2015 10:25:11 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 10:15, Jay K wrote: > > This is very good progress. > > Thank you for your speed and patience. > > > > > > And the error is kind of minor and uninteresting. Yes, I understand, it > > is fatal. Everything should just work to completion. > > > > > > See IsDarwin here: > > > > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > > > > cvsup is fiddling around how to use zlib. > > See here: > > https://github.com/modula3/cm3/blob/master/m3-tools/cvsup/suplib/src/m3makefile > > > > > > and surely cvsup isn't interesting any more? Who is using cvs? > > NetBSD and OpenBSD for starters. > > > > > You can delete m3-tools/cvsup. > > or change the scripts to skip it > > or update cm3cfg.common to be more up to date. > > I'd like to keep it, it's the best source for cvsup. > > I thought make-dist.py was just going to bootstrap (e.g. minimal > required). Is it actually building everything? > > > I was intending to run do-cm3-min.sh buildship, do-cm3-std.sh buildship, > do-cm3-caltech-parser.sh buildship after make-dist.py completed. > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 11:11:06 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:11:06 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1ED3.3000303@marino.st> References: <556700B0.8060500@marino.st>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> Message-ID: <556C21AA.70501@marino.st> On 6/1/2015 10:58, John Marino wrote: > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > hopefully that will work. I don't know quake language. quake doesn't know "False". If "0" doesn't work, I'll have to break down and come up with a patch to remove these last 3 lines. :) From jay.krell at cornell.edu Mon Jun 1 11:13:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 09:13:41 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C21AA.70501@marino.st> References: <556700B0.8060500@marino.st>, , <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, ,,<556885FD.6010800@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, ,,<55689065.1040207@marino.st>, ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, ,, <556B213E.8030108@marino.st>,,, , ,, <556B4BE9.1000902@marino.st>,,, , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, ,,, ,,<556C1218.3000103@marino.st> , , <556C16E7.505@marino.st> , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st> Message-ID: try FALSE Here is some sample code: https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common A patch shouldn't be needed but... hopefully soon I can try to duplicate what you are doing.. - Jay > Date: Mon, 1 Jun 2015 11:11:06 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 10:58, John Marino wrote: > > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > > hopefully that will work. I don't know quake language. > > quake doesn't know "False". If "0" doesn't work, I'll have to break > down and come up with a patch to remove these last 3 lines. :) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Mon Jun 1 11:24:56 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 1 Jun 2015 11:24:56 +0200 (CEST) Subject: [M3devel] duplicate subscription In-Reply-To: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> References: <20150601100213.9621487fcaea4a6e51e98fff@elegosoft.com> Message-ID: On Mon, 1 Jun 2015, Olaf Wagner wrote: > On Mon, 1 Jun 2015 09:11:22 +0200 (CEST) > Henning Thielemann wrote: > >> >> It seems that I am subscribed twice to this list. How can I find out the >> according e-mail addresses? > > Mailing list information can be found at > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel I know how to unsubscribe, but in order to do so I have to know the precise e-mail address I am subscribed with. I don't know them. :-( From adacore at marino.st Mon Jun 1 11:26:37 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:26:37 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , , <55689065.1040207@marino.st>, , , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , <556B213E.8030108@marino.st>, , , , , , <556B4BE9.1000902@marino.st>, , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , <556C1218.3000103@marino.st> , , <556C16E7.505@marino.st> , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st> Message-ID: <556C254D.7070306@marino.st> On 6/1/2015 11:13, Jay K wrote: > try FALSE > > Here is some sample code: > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > A patch shouldn't be needed but... hopefully soon I can try to duplicate > what you are doing.. Okay, I'll try "FALSE". That cm3cfg.common is pretty much what was installed at $STAGE/cm3-(all|min)-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/config/cm3cfg.common so I don't know why IsDarwin is not defined. === wait === okay, it failed again, same error (doesn't know IsDarwin), line 49 of m3-tools/cvsup/suplib/src/m3makefile So it seems we need to figure out why IsDarwin isn't known.... John From jay.krell at cornell.edu Mon Jun 1 11:29:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 09:29:54 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C254D.7070306@marino.st> References: <556700B0.8060500@marino.st>, ,,<20150529180223.d467621b68ebf665da685b12@elego.de>, , ,,<55689065.1040207@marino.st>, , ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , ,, <556B213E.8030108@marino.st>,,, , , , ,, <556B4BE9.1000902@marino.st>,,, , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , ,,, , , , <556C1218.3000103@marino.st> , ,,<556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> Message-ID: can you do like: export PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH cd /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib cm3 -commands -verbose cm3 -commands -verbose realclean cm3 -commands -verbose build - Jay > Date: Mon, 1 Jun 2015 11:26:37 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:13, Jay K wrote: > > try FALSE > > > > Here is some sample code: > > > > https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common > > > > A patch shouldn't be needed but... hopefully soon I can try to duplicate > > what you are doing.. > > Okay, I'll try "FALSE". > That cm3cfg.common is pretty much what was installed at > $STAGE/cm3-(all|min)-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/config/cm3cfg.common > so I don't know why IsDarwin is not defined. > > === wait === > > okay, it failed again, same error (doesn't know IsDarwin), line 49 of > m3-tools/cvsup/suplib/src/m3makefile > > So it seems we need to figure out why IsDarwin isn't known.... > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Mon Jun 1 11:44:02 2015 From: adacore at marino.st (John Marino) Date: Mon, 01 Jun 2015 11:44:02 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , <556B213E.8030108@marino.st>, , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , <556C1218.3000103@marino.st> , , , <556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> Message-ID: <556C2962.3020505@marino.st> On 6/1/2015 11:29, Jay K wrote: > can you do like: > export > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > cd > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > cm3 -commands -verbose > cm3 -commands -verbose realclean > cm3 -commands -verbose build It did not like "realclean" or "build": http://leaf.dragonflybsd.org/~marino/cvsupcmd.log John From jay.krell at cornell.edu Mon Jun 1 16:38:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 14:38:44 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C2962.3020505@marino.st> References: <556700B0.8060500@marino.st>, , ,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , , ,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , ,, <556B213E.8030108@marino.st>,,, , , , , , ,, <556B4BE9.1000902@marino.st>,,, , , , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , ,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , ,,, , , , , <556C1218.3000103@marino.st> , , ,,<556C16E7.505@marino.st> ,,, <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st> Message-ID: Sorry, multiple mistakes there by me. But I think I see the problem maybe. See it is always running mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 ? and never /mech/construction/mech/ptrees/default/lang/modula3/work/intermediate/compiler_with_{previous,self}/bin/{cm3,cm3cg} ? Do you have $CM3 set? Don't do that. It is supposed to be updating $PATH as it goes and finding cm3 and maybe cm3cg from there. But an environment variable CM3 will break it. https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py the function Setup That should print more, and maybe run and print the output of "where/which cm3" and "where/which cm3cg". and: https://github.com/modula3/cm3/blob/master/scripts/python/pylib.py: CM3 = ConvertPathForPython(getenv("CM3")) or "cm3" CM3 = SearchPath(CM3) I really just need to get and try your automation. You've made it available already? I'll have FreeBSD running locally "soon".. The right request by me would be to set PATH and CM3_INSTALL appropriately as make-dist.py would and run just building cvsup with -commands -verbose. Can you see how to do that? - Jay > Date: Mon, 1 Jun 2015 11:44:02 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:29, Jay K wrote: > > can you do like: > > export > > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > > cd > > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > > > cm3 -commands -verbose > > cm3 -commands -verbose realclean > > cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Jun 1 18:40:30 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 11:40:30 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C2962.3020505@marino.st> References: <556700B0.8060500@marino.st>, , , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , <556B213E.8030108@marino.st>, , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , <556C1218.3000103@marino.st> , , , <556C16E7.505@marino.st> , , <556C1ED3.3000303@marino.st>, <556C21AA.70501@marino.st>, , <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> Message-ID: <556C8AFE.1050204@lcwb.coop> I am getting ready to write more on this, but a quick response: For the do-cm3-*.sh scripts, the options are realclean, build, buildship, etc. For cm3, the (supposedly) corresponding options are -realclean, -build, -buildship. This burns me every time I switch from one command form to the other. On 06/01/2015 04:44 AM, John Marino wrote: > On 6/1/2015 11:29, Jay K wrote: >> can you do like: >> export >> PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH >> cd >> /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib >> >> cm3 -commands -verbose >> cm3 -commands -verbose realclean >> cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 19:00:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 12:00:00 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C1ED3.3000303@marino.st> References: <556700B0.8060500@marino.st>, <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de>, , <556885FD.6010800@marino.st>, , <20150529180223.d467621b68ebf665da685b12@elego.de>, , <55689065.1040207@marino.st>, , <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , <556B213E.8030108@marino.st>, , , , <556B4BE9.1000902@marino.st>, , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , <556C1218.3000103@marino.st> , <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> Message-ID: <556C8F90.9030907@lcwb.coop> On 06/01/2015 03:58 AM, John Marino wrote: > On 6/1/2015 10:53, Jay K wrote: >> make-dist makes everything. >> It makes a "min" distribution and an "all" distribution. >> Hopefully you can read it through it makes semi-reasonable sense. > > yes, I already figured this out. > >> This is all my doing -- which is to say, I quite like it, >> but not necessarily anyone else. >> min as I recall, is just the compiler: cm3, cm3cg, mklib, config, >> m3core, libm3 >> (mklib is only for NT386 target, but it is small and innocuous) >> all is everything, including min >> I don't like the confusion and purported complexity of breaking >> things down into several medium sized packages, with dependencies. >> I want more of a "one stop shop", granted, a big one, that >> gives everyone stuff they won't use. >> But I compromised and it makes to things. > > > Right now, I'm assuming I can write an "install target" that copies from > $STAGE/$DISTRIBUTION to where it needs to be installed in ports. If so, > this approach is fine. Simpler even. > > >> >> cvs is pretty awful. >> I'm still more productive in it than git, but I expect that will change. >> But ok. cvsup can work. The error is for a trivial thing. > > I'm attempting to replace "IsDarwin() or IsSolaris()" with "False" and > hopefully that will work. I don't know quake language. > Replace them with "", empty string. > (I don't know why IsDarwin does't work because it's defined in the new > cm3 distibution's cm3cfg.common.) > Yes, that surprises me too. suggests the new cm3cfg.common isn't getting interpreted. > Let's see how it goes. > John > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 1 19:48:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 12:48:34 -0500 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , <556BD14D.5050507@lcwb.coop> Message-ID: <556C9AF2.7090704@lcwb.coop> On 06/01/2015 01:40 AM, Jay K wrote: > Yes, exactly, that's what I was trying to say, in my later/last mail. > > > Yes the compability is definitely two-way. > There is a one to one mapping back and forth. > cm3 goes with cm3cg; cm3cg goes with cm3. > Neither is compatible with older nor necessarily newer versions. > > > They would be statically linked together if not for the GPL. > > > When I first placed m3cc in pkginfo.txt years ago, I was only thinking > of build order and, like, "override". > > > m3cc imports nothing and nothing imports m3cc. > In the Modula-3 module/library/interface sense. > > > It can be built when you have nothing. > So I figured put it nearly first. > > > Only now today did I consider the shipping order. > It should be shipped adjacent to cm3, either right before or right after. > > > If you split build and ship, then it can still be built early, > or at almost any time. > Yes, this is the way I want to do building. First build everything you want to without putting any of it into use as build tools. Then put it all into use. One annoyance we currently have is that the do-cm3.*.sh scripts have, perhaps we should call them subcommands, like realclean, build, ship, buildship, etc. But the cm3 command treats their counterparts as command line options, -realclean, -build, -ship, -buildship, etc. I would like to consistify this. Removing the hyphen from cm3 would create ambiguities with file names, so it would mean adding the hyphen to the scripts. Much more serious is that similarly-named options do not have the same meaning in the scripts and in cm3. cm3 -build followed by cm3 -ship works as I expect, but do-cm3-*.sh build causes me nothing but trouble, and I learned long ago to avoid it. Instead, I always use do-cm3-*.sh buildship. This ships almost everything as it is built, but usually only cm3 and cm3cg executables matter, and today, they are specifically excepted from shipping, if you do not set INSTALL_CM3_IN_BIN="yes". I am writing from memory, but I think the problem with do-cm3-*.sh build is that build also uses overrides, which I barely understand, except that they, at best make things unshippable, and I think there was some other problem too. If we made the override thing a separate option to the scripts, so just build would neither ship nor override, I think that would satisfy my need, at least for the time being. And it would be consistent. Actually, I think we do want to use just-rebuilt packages when linking to them. Only executables and things they link in (possibly dynamically) need to be left alone until a later ship step. This usually only means cm3, cm3cg, and, if dynamically linked in, m3core. But there are others to think about, e.g., m3bundle, stubgen, netobjgen. > > But we have one ordering for build and ship, so let the shipping order decide. > > > I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. I have no objection to addressing the copy-over-an-executing-executable problem in a different way, and no objection to getting rid of the environment variable. But I do want it, for now at least, to be possible to get it to not ship either cm3 or cm3cg, even with the buildship option. If it just unconditionally doesn't ship them, that is fine. Before I added the variable check for the cm3cg package, it was unconditionally shipping it. > I still think it broke upgrade.py. > I can workaround. But I'd like to remove the environment variable check. > > > There are many order dependencies in build and ship. This is just one. > > > - Jay > > > > > > Date: Sun, 31 May 2015 22:28:13 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; adacore at marino.st > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > > to fail regardless of the order of building the two. > > > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > > it should work, because m3cc contains no Modula-3 code, and the new > > cm3 would not, when building it, run cm3cg. After that, there would > > be a consistent set. for further compiles. But any package containing > > M3 code built between cm3 and m3cc would fail. > > > > On 05/31/2015 05:15 PM, Jay wrote: > > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > > > - Jay > > > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > >> Should I put it back? From the last response I'm guessing that wasn't > > >> the problem. > > >> > > >> John > > >> > > >> > > >> On 5/31/2015 19:09, Jay K wrote: > > >>> Oh probably the problem is using the backend premature actually. > > >>> Darn, I can't help right now. > > >>> > > >>> - Jay > > >>> > > >>> > > >>> > > >>> ------------------------------------------------------------------------ > > >>> From: jay.krell at cornell.edu > > >>> To: adacore at marino.st; m3devel at elegosoft.com > > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > >>> (0x100), expected 0x110 > > >>> > > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > > >>> but my read was it didn't try that. > > >>> So.. with INSTALL_CM3_IN_BIN=yes? > > >>> I need to try this all, and "fix" it to set that. The whole thing in > > >>> m3cc/src/m3makefile is relatively > > >>> new vs. when I was actively using all this. :( > > >>> > > >>> - Jay > > >>> > > >>> > > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > > >>>> From: adacore at marino.st > > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > >>> (0x100), expected 0x110 > > >>>> > > >>>> On 5/31/2015 11:53, Jay K wrote: > > >>>>> John, have you tried make-dist.py? > > >>>>> i.e. > > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > > >>>>> > > >>>>> While it might not be exactly what you need, it should demonstrate the > > >>>>> elements. > > >>>>> > > >>>>> I will try to try this all soon, really, I hope so. > > >>>>> > > >>>>> It starts with a working compiler, does no in-place updates, build into > > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > > >>>>> It does "min" and "all". > > >>>>> > > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > > >>>>> > > >>>>> - Jay > > >>>> > > >>>> Hi Jay, > > >>>> I'm getting the same error as always using make-dist script: > > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > > >>>> > > >>>> Thanks, > > >>>> John > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 1 21:47:48 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 1 Jun 2015 19:47:48 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556C9AF2.7090704@lcwb.coop> References: <556700B0.8060500@marino.st>,<55682D1A.2030504@marino.st> <20150529113935.7b0ebcf1bb584c5a8543e0b5@elego.de> <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>,,<556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , , , <556BD14D.5050507@lcwb.coop>, , <556C9AF2.7090704@lcwb.coop> Message-ID: Sorry I'm rushing here..: Imho, build should build and ship should ship and buildship should do both. If that isn't the case, we should fix. Then, if you don't want to ship something, don't ship it. Not, ship it and set the environment variable. >implied: What is override? There are essentially two package pools, or sets of libraries, to chose from. They are roughly speaking the installed/shipped stuff: /usr/local/cm3/pkg/m3core/target (obviously it can vary) and the locally built but not yet shipped stuff, I call them: $HOME/dev2/cm3/m3-libs/m3core/target (obviously it can vary) I also call this "the object tree", i.e. the tree with a bunch of .obj/.o files. Or the "intermediates" tree, i.e. a bunch of outputs that are pruned out by "setup", and the outputs that are ultimately interesting The default is the first, override means to use the second. Then, the idea is that only stuff built against the install is safe to overwrite/add to the install. But I think is missing there is that, well, if I've built the entire system with overrides and I'm going to ship the entire system, then ship is safe. This isn't the best approach imho. Arguably better approaches are: always have a 2-level search first the object tree then the install tree This breaks people who switch around between incompatible changes in m3core and random application development, with the same source/object tree. However maybe that is just tough. The object tree doesn't have to be in the source tree. You should be able to say, like: symlinktree /usr/local/cm3 /working-on-feature1 export/set CM3_SOMETHING=/working-on-feature1 and case I change cm3 also export/set PATH=/working-on-feature1/bin:$PATH and then copy on write as you ship and then to switch to some other work: symlinktree /usr/local/cm3 /working-on-feature2 export/set CM3_SOMETHING=/working-on-feature2 and case I change cm3 also export/set PATH=/working-on-feature2/bin:$PATH and then copy on write as you ship, different stuff That is a working model that I advocate but don't yet follow. Though make-dist kind of does. make-dist copies the compiler to the new location and then does buildship. In this model, you don't have overrides and you always build/ship. And if you mess up, you can start over easily. But I haven't actually established workflow here so I'm just talking too much. : A real analog in other people's workflows is: mkdir /obj/working-on-feature1 cd /obj/working-on-feature1 /src/configure ... make mkdir /obj/working-on-feature2 cd /obj/working-on-feature2 /src/configure ...potentially different .. make .sh build should not ship. Agreed. Currently if you are going to ship, you can't override. make-dist.py uses a destdir-like approach. There is some contention perhaps of scenarios of "I'm just building a little, small safe changes" and "I'm rebuilding everything, and possibly messing up". I really want the environment variable checks to go away. - Jay > Date: Mon, 1 Jun 2015 12:48:34 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > On 06/01/2015 01:40 AM, Jay K wrote: > > Yes, exactly, that's what I was trying to say, in my later/last mail. > > > > > > Yes the compability is definitely two-way. > > There is a one to one mapping back and forth. > > cm3 goes with cm3cg; cm3cg goes with cm3. > > Neither is compatible with older nor necessarily newer versions. > > > > > > They would be statically linked together if not for the GPL. > > > > > > When I first placed m3cc in pkginfo.txt years ago, I was only thinking > > of build order and, like, "override". > > > > > > m3cc imports nothing and nothing imports m3cc. > > In the Modula-3 module/library/interface sense. > > > > > > It can be built when you have nothing. > > So I figured put it nearly first. > > > > > > Only now today did I consider the shipping order. > > It should be shipped adjacent to cm3, either right before or right after. > > > > > > If you split build and ship, then it can still be built early, > > or at almost any time. > > > > Yes, this is the way I want to do building. First build everything you want to > without putting any of it into use as build tools. Then put it all into use. > > One annoyance we currently have is that the do-cm3.*.sh scripts have, perhaps > we should call them subcommands, like realclean, build, ship, buildship, etc. > But the cm3 command treats their counterparts as command line options, > -realclean, -build, -ship, -buildship, etc. I would like to consistify > this. Removing the hyphen from cm3 would create ambiguities with file names, > so it would mean adding the hyphen to the scripts. > > Much more serious is that similarly-named options do not have the same meaning > in the scripts and in cm3. cm3 -build followed by cm3 -ship works as I expect, > but do-cm3-*.sh build causes me nothing but trouble, and I learned long ago to > avoid it. Instead, I always use do-cm3-*.sh buildship. This ships almost everything > as it is built, but usually only cm3 and cm3cg executables matter, and today, they are > specifically excepted from shipping, if you do not set INSTALL_CM3_IN_BIN="yes". > > I am writing from memory, but I think the problem with do-cm3-*.sh build is that > build also uses overrides, which I barely understand, except that they, at best > make things unshippable, and I think there was some other problem too. If we > made the override thing a separate option to the scripts, so just build would > neither ship nor override, I think that would satisfy my need, at least for the > time being. And it would be consistent. > > Actually, I think we do want to use just-rebuilt packages when linking to them. > Only executables and things they link in (possibly dynamically) need to be > left alone until a later ship step. This usually only means cm3, cm3cg, > and, if dynamically linked in, m3core. But there are others to think about, > e.g., m3bundle, stubgen, netobjgen. > > > > > But we have one ordering for build and ship, so let the shipping order decide. > > > > > > I still don't like INSTALL_CM3_IN_BIN and I'd still like to remove it. > > I have no objection to addressing the copy-over-an-executing-executable problem > in a different way, and no objection to getting rid of the environment variable. > But I do want it, for now at least, to be possible to get it to not ship either > cm3 or cm3cg, even with the buildship option. If it just unconditionally doesn't > ship them, that is fine. Before I added the variable check for the cm3cg package, > it was unconditionally shipping it. > > > I still think it broke upgrade.py. > > I can workaround. But I'd like to remove the environment variable check. > > > > > > There are many order dependencies in build and ship. This is just one. > > > > > > - Jay > > > > > > > > > > > Date: Sun, 31 May 2015 22:28:13 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; adacore at marino.st > > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > > > I am quite sure that the cm3/cm3cg incompatibility is two-way, i.e., > > > both must be new or both old. So, I would expect INSTALL_CM3_IN_BIN=yes > > > to fail regardless of the order of building the two. > > > > > > OTOH, if cm3 were built and shipped, then m3cc came immediately after, > > > it should work, because m3cc contains no Modula-3 code, and the new > > > cm3 would not, when building it, run cm3cg. After that, there would > > > be a consistent set. for further compiles. But any package containing > > > M3 code built between cm3 and m3cc would fail. > > > > > > On 05/31/2015 05:15 PM, Jay wrote: > > > > I think now it might not be the problem. Could be m3cc needs to be later in pkginfo.txt. I have to try later. > > > > > > > > - Jay > > > > > > > > On May 31, 2015, at 10:59 AM, John Marino wrote: > > > > > > > >> I removed "INSTALL_CM3_IN_BIN=yes" after reading the thread about it. > > > >> Should I put it back? From the last response I'm guessing that wasn't > > > >> the problem. > > > >> > > > >> John > > > >> > > > >> > > > >> On 5/31/2015 19:09, Jay K wrote: > > > >>> Oh probably the problem is using the backend premature actually. > > > >>> Darn, I can't help right now. > > > >>> > > > >>> - Jay > > > >>> > > > >>> > > > >>> > > > >>> ------------------------------------------------------------------------ > > > >>> From: jay.krell at cornell.edu > > > >>> To: adacore at marino.st; m3devel at elegosoft.com > > > >>> Date: Sun, 31 May 2015 16:45:42 +0000 > > > >>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > > >>> (0x100), expected 0x110 > > > >>> > > > >>> Hm. Strange. I need to read more closely. I see it skipping shipping, > > > >>> but my read was it didn't try that. > > > >>> So.. with INSTALL_CM3_IN_BIN=yes? > > > >>> I need to try this all, and "fix" it to set that. The whole thing in > > > >>> m3cc/src/m3makefile is relatively > > > >>> new vs. when I was actively using all this. :( > > > >>> > > > >>> - Jay > > > >>> > > > >>> > > > >>>> Date: Sun, 31 May 2015 16:57:02 +0200 > > > >>>> From: adacore at marino.st > > > >>>> To: jay.krell at cornell.edu; m3devel at elegosoft.com > > > >>>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp > > > >>> (0x100), expected 0x110 > > > >>>> > > > >>>> On 5/31/2015 11:53, Jay K wrote: > > > >>>>> John, have you tried make-dist.py? > > > >>>>> i.e. > > > >>> https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py > > > >>>>> > > > >>>>> While it might not be exactly what you need, it should demonstrate the > > > >>>>> elements. > > > >>>>> > > > >>>>> I will try to try this all soon, really, I hope so. > > > >>>>> > > > >>>>> It starts with a working compiler, does no in-place updates, build into > > > >>>>> a unique directory in /tmp, and gives you .tar.gz or such at the end. > > > >>>>> It does "min" and "all". > > > >>>>> > > > >>>>> If you set the STAGE environment variable, it uses that instead of /tmp. > > > >>>>> It should probably be called M3_STATE or M3_MAKEDIST_STAGE. > > > >>>>> > > > >>>>> - Jay > > > >>>> > > > >>>> Hi Jay, > > > >>>> I'm getting the same error as always using make-dist script: > > > >>>> http://leaf.dragonflybsd.org/~marino/m3d.log > > > >>>> > > > >>>> Thanks, > > > >>>> John > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Jun 2 00:16:57 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 1 Jun 2015 18:16:57 -0400 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> Message-ID: <20150601221657.GA4281@topoi.pooq.com> On Mon, Jun 01, 2015 at 07:47:48PM +0000, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. Make sense to me. -- hendrik From hendrik at topoi.pooq.com Tue Jun 2 00:23:19 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 1 Jun 2015 18:23:19 -0400 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> Message-ID: <20150601222319.GA4410@topoi.pooq.com> On Mon, Jun 01, 2015 at 07:47:48PM +0000, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. This would mean there would have to be some kind of loading dok for things that have been built but not yet shipped. It probably exists already. -- hendrik From rodney_bates at lcwb.coop Tue Jun 2 02:54:45 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Jun 2015 19:54:45 -0500 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, <55683689.2090709@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st>, , <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st>, , , , <556BD14D.5050507@lcwb.coop>, , <556C9AF2.7090704@lcwb.coop> Message-ID: <556CFED5.80002@lcwb.coop> Here is a short-term proposal (i.e., without major reorganization) for the do-cm3*.sh scripts: 1) 'build' only builds, as we seem to agree it should. 2) a new option 'override' (and only 'override') causes an override build 3) a new option 'partialship' ships, as each package is done, things that will be needed to compile another package that does a quake import on the just-built package (I think this means static library, if any, and .M3WEB), but does not ship things that will be used to execute the just-built package (I think this means executable or dynamic library). I'm not sure right off hand which ship group things like interface source files, etc. belong in. Plus, just as conveniences: 4) buildship means the same as build and ship (I think this is already the case.) 5) buildpartialship means the same as build and partialship 6) All these options at least allow a leading hyphen, so they are like the cm3 command. I guess cm3 should also have a -shippartial On 06/01/2015 02:47 PM, Jay K wrote: > Sorry I'm rushing here..: > > > Imho, build should build and ship should ship and buildship should do both. > If that isn't the case, we should fix. > Then, if you don't want to ship something, don't ship it. > Not, ship it and set the environment variable. > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 2 09:11:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 2 Jun 2015 07:11:44 +0000 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, ,,,,<20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com>, , ,,,,<5568B352.8060005@marino.st>, <556ABB1C.5080607@marino.st>, , , , , , , <556B213E.8030108@marino.st>, , , , ,,, , , , , , , <556B4BE9.1000902@marino.st>, , , , ,,, , ,,,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , ,,,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , ,,,,<556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , ,,,,, , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: John, this and your other problems should all be fixed now. I was able to reproduce them mostly.I didn't actually reproduce the cvsup problem, but I see in the log you did set CM3, and I did test with that, and hit the cm3cg version mismatch, and the problem is pretty clear -- my mistake. See: https://github.com/modula3/cm3/commit/a149d4b5e715d89b076c92e0b41d74f9b5ff56e4 https://github.com/modula3/cm3/commit/769784562d0e45b968e515c1412ed9247ec050ef https://github.com/modula3/cm3/commit/a0e44b08e687a8bba677c299ffa459993525944c Thank you for your patience and diligence. I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && make-dist.py or skip right to make-dist.py.There was an additional problem skipping right to make-dist.py that I fixed, at least for Darwin. I don't use any of the .sh files -- I don't want to read/maintain/learn that programming language as I suspect it isn't particularly good. - Jay From: jay.krell at cornell.edu To: adacore at marino.st CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 Date: Mon, 1 Jun 2015 14:38:44 +0000 Sorry, multiple mistakes there by me. But I think I see the problem maybe. See it is always running mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 ? and never /mech/construction/mech/ptrees/default/lang/modula3/work/intermediate/compiler_with_{previous,self}/bin/{cm3,cm3cg} ? Do you have $CM3 set? Don't do that. It is supposed to be updating $PATH as it goes and finding cm3 and maybe cm3cg from there. But an environment variable CM3 will break it. https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py the function Setup That should print more, and maybe run and print the output of "where/which cm3" and "where/which cm3cg". and: https://github.com/modula3/cm3/blob/master/scripts/python/pylib.py: CM3 = ConvertPathForPython(getenv("CM3")) or "cm3" CM3 = SearchPath(CM3) I really just need to get and try your automation. You've made it available already? I'll have FreeBSD running locally "soon".. The right request by me would be to set PATH and CM3_INSTALL appropriately as make-dist.py would and run just building cvsup with -commands -verbose. Can you see how to do that? - Jay > Date: Mon, 1 Jun 2015 11:44:02 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > On 6/1/2015 11:29, Jay K wrote: > > can you do like: > > export > > PATH=$STAGE/cm3-all-AMD64_FREEBSD-d5.10.0-FreeBSD10.1-20150601/bin/:$PATH > > cd > > /mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-tools/cvsup/suplib > > > > cm3 -commands -verbose > > cm3 -commands -verbose realclean > > cm3 -commands -verbose build > > It did not like "realclean" or "build": > http://leaf.dragonflybsd.org/~marino/cvsupcmd.log > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Tue Jun 2 09:19:33 2015 From: adacore at marino.st (John Marino) Date: Tue, 02 Jun 2015 09:19:33 +0200 Subject: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>, , , , , , , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: <556D5905.4010304@marino.st> On 6/2/2015 09:11, Jay K wrote: > John, this and your other problems should all be fixed now. > > I was able to reproduce them mostly. > I didn't actually reproduce the cvsup problem, but I see in the log you > did set CM3, and I did test with that, and hit the cm3cg version > mismatch, and the problem is pretty clear -- my mistake. Thanks Jay! Don't worry about the mistake. I'm just happy that my use case was able to flush out some issues and get some interesting discussion going. Sure it's been hard-going from my side but the final result is an improvement for everyone. I'll see what I can test at night in a VM but I don't get home back to my "real" machine until late Friday night (CEDT). But after looking over the commits this seems promising. John From wagner at elegosoft.com Tue Jun 2 11:36:11 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 2 Jun 2015 11:36:11 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556CFED5.80002@lcwb.coop> References: <556700B0.8060500@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> Message-ID: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> On Mon, 01 Jun 2015 19:54:45 -0500 "Rodney M. Bates" wrote: > Here is a short-term proposal (i.e., without major reorganization) > for the do-cm3*.sh scripts: > > 1) 'build' only builds, as we seem to agree it should. > 2) a new option 'override' (and only 'override') causes an override build > 3) a new option 'partialship' ships, as each package is done, things that > will be needed to compile another package that does a quake import on the > just-built package (I think this means static library, if any, and .M3WEB), > but does not ship things that will be used to execute the just-built package > (I think this means executable or dynamic library). I'm not sure right off > hand which ship group things like interface source files, etc. belong in. I don't know how you will implement this and how it should fit into the M3 package concept, unless you ship to a completely different location, i.e. another package pool (but cm3 currently just supports one). In my opinion the package system relies on the invariant that a shipped (installed) package is completely consistent, i.e. the source code interfaces correspond to the intermediate code interfaces and to the binary equivalent of them (static and dynamic libraries). If I understand you correctly, you want to give up this consistency in favour of being able to do an easier bootstrap of the CM3 system. But bootstrapping a complex system is never easy and usually requires some special or tricky steps. Ordinary use of the system, i.e. all other applications, that just build on the standard tools, don't need this kind of steps. If I could design (and implement) a better cm3 builder, I'd have one with multiple package pools (shipping destinations, locations of installed packages) and an integrated builder that knows all about the package dependencies (so that we haven't to do that in scripts). The two points are the only shortcomings in the cm3 build system in my opinion. The prjm tools of the ComPact sources was a step in this direction: it can handle multiple pools of packages and their dependencies, but is language independent, therefore too complex, and never got integrated into the compiler builder. As for the shell scripts, I'm surprised that they have survived so long; they just got created on the fly to support the building and publication of the CM3 system as open source when we got the sources from Critical Mass. They were never intended as a general user interface, but only as tools for the CM3 maintainers, and later got used in the Hudson CI. BTW, if I find some time and access to suitable machines again, I'd really like to set up a new CI system on Jenkins interfacing with the Github infrastruture. But I won't make any promises here. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From lists at darko.org Tue Jun 2 14:05:48 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 2 Jun 2015 05:05:48 -0700 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> References: <556700B0.8060500@marino.st> <20150529122732.1396eb079ff9b404bcb21e40@elego.de> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> Message-ID: Why can't we design and build a cm3 builder that supports these requirements? Relying on scripts seems entirely broken and an anathema to the portability of the CM3 system. There seems to be enough experience to specify and design such a system and building it doesn't sound like it would be very difficult. I'm sure there would be enough volunteers to implement it since people's productivity is obviously being affected. Wouldn't we want to fix the problem for once and for all instead of endlessly hacking scripts? Having a robust, reliable and portable end-to-end build system would also make it easier for people wanting to port and enhance the compiler, which would be a boost for the project. On Tue, Jun 2, 2015 at 2:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > > > Here is a short-term proposal (i.e., without major reorganization) > > for the do-cm3*.sh scripts: > > > > 1) 'build' only builds, as we seem to agree it should. > > 2) a new option 'override' (and only 'override') causes an override build > > 3) a new option 'partialship' ships, as each package is done, things that > > will be needed to compile another package that does a quake import > on the > > just-built package (I think this means static library, if any, and > .M3WEB), > > but does not ship things that will be used to execute the just-built > package > > (I think this means executable or dynamic library). I'm not sure > right off > > hand which ship group things like interface source files, etc. > belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other > applications, > that just build on the standard tools, don't need this kind of steps. > > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 2 18:18:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 02 Jun 2015 11:18:03 -0500 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> Message-ID: <556DD73B.9050601@lcwb.coop> On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org From dmuysers at hotmail.com Tue Jun 2 18:39:35 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 2 Jun 2015 18:39:35 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556DD73B.9050601@lcwb.coop> References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop><20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: golang is an example of good package management. It`s interaction of the builder with github repositories is an interesting idea. -----Original Message----- From: Rodney M. Bates Sent: Tuesday, June 02, 2015 6:18 PM To: Olaf Wagner Cc: m3devel Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 2 19:48:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 2 Jun 2015 17:48:13 +0000 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556700B0.8060500@marino.st> <5568421D.3050209@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop><20150602113611.66de95acf8cdb242c6672605@elegosoft.com>, <556DD73B.9050601@lcwb.coop>, Message-ID: > Yes, absolutely. We need a greatly reworked build system for multiple packages. > For building within a package, cm3's existing build is really quite sophisticated. Exactly my thoughts for years. quake is nice and declarative for each package, and currently does nothing for multiple packages. How far would we get with the following: include_dir to stitch together packages, the same as it stitches directories within packages. "Flush" the state at each Library() or Program() invocation? Library and Program have to be last currently? So that would work? Or they can be anywhere? Furthermore, target directory either be: 1) ./target -- might be what it'd do now, but is wrong choice 2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. cd anywhere, that is still it 3) under some new new root, like go up from cwd until you find special marker file; this would be building outside source tree entirely 4) same as currently, wherever is Library()/Program(), go ../ If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"? Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir. it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib. It looks like:dirs file: DIRS=a \ b \ c sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake I use this system a lot and quite like it.Though it is obscure. The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad. I think scons works this way too, the language being Python. Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program. Make sense? Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.) - Jay From: dmuysers at hotmail.com To: rodney.m.bates at acm.org; wagner at elegosoft.com Date: Tue, 2 Jun 2015 18:39:35 +0200 CC: m3devel at elegosoft.com Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 golang is an example of good package management. It`s interaction of the builder with github repositories is an interesting idea. -----Original Message----- From: Rodney M. Bates Sent: Tuesday, June 02, 2015 6:18 PM To: Olaf Wagner Cc: m3devel Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 On 06/02/2015 04:36 AM, Olaf Wagner wrote: > On Mon, 01 Jun 2015 19:54:45 -0500 > "Rodney M. Bates" wrote: > >> Here is a short-term proposal (i.e., without major reorganization) >> for the do-cm3*.sh scripts: >> >> 1) 'build' only builds, as we seem to agree it should. >> 2) a new option 'override' (and only 'override') causes an override build >> 3) a new option 'partialship' ships, as each package is done, things that >> will be needed to compile another package that does a quake import on the >> just-built package (I think this means static library, if any, and .M3WEB), >> but does not ship things that will be used to execute the just-built package >> (I think this means executable or dynamic library). I'm not sure right off >> hand which ship group things like interface source files, etc. belong in. > > I don't know how you will implement this and how it should fit into the > M3 package concept, unless you ship to a completely different location, > i.e. another package pool (but cm3 currently just supports one). > > In my opinion the package system relies on the invariant that a shipped > (installed) package is completely consistent, i.e. the source code > interfaces correspond to the intermediate code interfaces and to the > binary equivalent of them (static and dynamic libraries). If I understand > you correctly, you want to give up this consistency in favour of being > able to do an easier bootstrap of the CM3 system. Yes, but we already must have a temporarily inconsistent state of installation today. This is a somewhat different kind of inconsistency, but inconsistent, nonetheless. I guess -partialship could do an additional final pass over the package list, doing a full ship. The inconsistent state would only last during a single user-requested step, which is about as short as it can get. My intent was that one would immediately do a full ship after a partialship, although as a separate command, thus restoring consistency. > > But bootstrapping a complex system is never easy and usually requires some > special or tricky steps. Ordinary use of the system, i.e. all other applications, > that just build on the standard tools, don't need this kind of steps. > And my proposal is a simple, short-term way to provide somewhat more flexible options for just such special or tricky steps. It will work the same way as now, if you don't use the new options, and would allow Jay to get rid of the hated INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > If I could design (and implement) a better cm3 builder, I'd have one with > multiple package pools (shipping destinations, locations of installed > packages) and an integrated builder that knows all about the package > dependencies (so that we haven't to do that in scripts). The two points > are the only shortcomings in the cm3 build system in my opinion. > The prjm tools of the ComPact sources was a step in this direction: > it can handle multiple pools of packages and their dependencies, but > is language independent, therefore too complex, and never got integrated > into the compiler builder. > Yes, absolutely. We need a greatly reworked build system for multiple packages. For building within a package, cm3's existing build is really quite sophisticated. I don't know of any other that detects the need for recompilation on declaration-granularity. But it does very little with inter-package problems -- just detection of link-time inconsistencies, with hopelessly uninformative error messages. (I once set out to improve them, but it required more info in the .mx and .m3x files, with more incompatible changes, and I only got one message improved a little bit, before I gave up for the time.) I agree, reworking it all is very desirable. It will take a lot of rework. I think we all agree that some kind of multi-layer, multi-destination scheme is needed. This will probably entail disentangling the source and build directories from being interspersed in each package, or something equally perturbing to the existing system. It isn't going to be a little patch. I think my proposal will be small to implement and have no impact on anything existing, if you don't use the new options. > As for the shell scripts, I'm surprised that they have survived so long; > they just got created on the fly to support the building and publication > of the CM3 system as open source when we got the sources from Critical > Mass. They were never intended as a general user interface, but only > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > BTW, if I find some time and access to suitable machines again, I'd > really like to set up a new CI system on Jenkins interfacing with the > Github infrastruture. But I won't make any promises here. > Yes, this would be very good. Priorities, priorities! :-(. > Olaf > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Jun 2 20:59:51 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 2 Jun 2015 14:59:51 -0400 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: References: <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: <20150602185951.GA6137@topoi.pooq.com> On Tue, Jun 02, 2015 at 05:48:13PM +0000, Jay K wrote: > > Yes, absolutely. We need a greatly reworked build system for multiple packages. > > For building within a package, cm3's existing build is really quite sophisticated. > > Exactly my thoughts for years. quake is nice and declarative for each package, and currently > does nothing for multiple packages. > > > How far would we get with the following: > > > include_dir to stitch together packages, the same as it stitches > directories within packages. > > "Flush" the state at each Library() or Program() invocation? > > Library and Program have to be last currently? So that would work? > Or they can be anywhere? > > > Furthermore, target directory either be: > 1) ./target -- might be what it'd do now, but is wrong choice Probably the wrong place. ./cm3target might be slightly better. Or make it not hidden. Or have a quake conmend to set it. > 2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. > cd anywhere, that is still it > 3) under some new new root, like go up from cwd until you find > special marker file; this would be building outside source tree entirely Outside the source tree entirely would be good. I'd really like to be able to compile a source tree that's on a read-only file system. Or on a shadow file system. > 4) same as currently, wherever is Library()/Program(), go ../ > > If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"? > > Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir. > > it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib. > > It looks like:dirs file: DIRS=a \ b \ c > sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake > I use this system a lot and quite like it.Though it is obscure. > The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad. > > I think scons works this way too, the language being Python. Writing the cm3 stanzas to tell scons how to use cm3 sould be useful, too. > > Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program. > > Make sense? > Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.) > > - Jay > > > > > From: dmuysers at hotmail.com > To: rodney.m.bates at acm.org; wagner at elegosoft.com > Date: Tue, 2 Jun 2015 18:39:35 +0200 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 > > > > > > golang is an example of good > package management. > It`s interaction of the builder with github repositories is an interesting > idea. > > -----Original Message----- > From: Rodney M. Bates > Sent: Tuesday, June 02, 2015 6:18 PM > To: Olaf Wagner > Cc: m3devel > Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG > version stamp (0x100), expected 0x110 > > > > On 06/02/2015 04:36 AM, Olaf Wagner wrote: > > On Mon, 01 Jun 2015 19:54:45 -0500 > > "Rodney M. Bates" wrote: > > > >> Here is a short-term proposal (i.e., without major > reorganization) > >> for the do-cm3*.sh scripts: > >> > >> 1) 'build' only builds, as we seem to agree it should. > >> 2) a new option 'override' (and only 'override') causes an > override build > >> 3) a new option 'partialship' ships, as each package is done, > things that > >> will be needed to compile another > package that does a quake import on the > >> just-built package (I think this > means static library, if any, and .M3WEB), > >> but does not ship things that will > be used to execute the just-built package > >> (I think this means executable or > dynamic library). I'm not sure right off > >> hand which ship group things like > interface source files, etc. belong in. > > > > I don't know how you will implement this and how it should fit into > the > > M3 package concept, unless you ship to a completely different > location, > > i.e. another package pool (but cm3 currently just supports one). > > > > In my opinion the package system relies on the invariant that a > shipped > > (installed) package is completely consistent, i.e. the source > code > > interfaces correspond to the intermediate code interfaces and to > the > > binary equivalent of them (static and dynamic libraries). If I > understand > > you correctly, you want to give up this consistency in favour of > being > > able to do an easier bootstrap of the CM3 system. > > Yes, but we already must have a temporarily inconsistent state of > installation > today. This is a somewhat different kind of inconsistency, but > inconsistent, > nonetheless. > > I guess -partialship could do an additional final pass over the package > list, > doing a full ship. The inconsistent state would only last during a > single > user-requested step, which is about as short as it can get. My intent > was > that one would immediately do a full ship after a partialship, although > as > a separate command, thus restoring consistency. > > > > > > But bootstrapping a complex system is never easy and usually requires > some > > special or tricky steps. Ordinary use of the system, i.e. all other > applications, > > that just build on the standard tools, don't need this kind of > steps. > > > > And my proposal is a simple, short-term way to provide somewhat more > flexible > options for just such special or tricky steps. It will work the same > way as now, > if you don't use the new options, and would allow Jay to get rid of the > hated > INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special > steps. > > > If I could design (and implement) a better cm3 builder, I'd have one > with > > multiple package pools (shipping destinations, locations of > installed > > packages) and an integrated builder that knows all about the > package > > dependencies (so that we haven't to do that in scripts). The two > points > > are the only shortcomings in the cm3 build system in my opinion. > > The prjm tools of the ComPact sources was a step in this > direction: > > it can handle multiple pools of packages and their dependencies, > but > > is language independent, therefore too complex, and never got > integrated > > into the compiler builder. > > > > Yes, absolutely. We need a greatly reworked build system for multiple > packages. > For building within a package, cm3's existing build is really quite > sophisticated. > I don't know of any other that detects the need for recompilation > on declaration-granularity. But it does very little with > inter-package > problems -- just detection of link-time inconsistencies, with > hopelessly > uninformative error messages. (I once set out to improve them, but > it > required more info in the .mx and .m3x files, with more incompatible > changes, and I only got one message improved a little bit, before I > gave > up for the time.) > > I agree, reworking it all is very desirable. It will take a lot of > rework. > I think we all agree that some kind of multi-layer, multi-destination > scheme is needed. This will probably entail disentangling the > source > and build directories from being interspersed in each package, or > something > equally perturbing to the existing system. It isn't going to be a > little > patch. I think my proposal will be small to implement and have no > impact > on anything existing, if you don't use the new options. > > > > As for the shell scripts, I'm surprised that they have survived so > long; > > they just got created on the fly to support the building and > publication > > of the CM3 system as open source when we got the sources from > Critical > > Mass. They were never intended as a general user interface, but > only > > as tools for the CM3 maintainers, and later got used in the Hudson > CI. > > > > BTW, if I find some time and access to suitable machines again, > I'd > > really like to set up a new CI system on Jenkins interfacing with > the > > Github infrastruture. But I won't make any promises here. > > > > Yes, this would be very good. Priorities, priorities! :-(. > > > Olaf > > > > -- > Rodney Bates > rodney.m.bates at acm.org From wagner at elegosoft.com Tue Jun 2 21:19:52 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 2 Jun 2015 21:19:52 +0200 Subject: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110 In-Reply-To: <556DD73B.9050601@lcwb.coop> References: <556700B0.8060500@marino.st> <20150529155833.8454664b88391d93708e8f85@elego.de> <55687266.3090400@marino.st> <20150529162028.388911007a614006049beb42@elego.de> <20150529172305.a3fc2a9865b5250a5b9140fc@elego.de> <556885FD.6010800@marino.st> <20150529180223.d467621b68ebf665da685b12@elego.de> <55689065.1040207@marino.st> <20150529201531.b35e3cfdafeb9e2735e0020b@elegosoft.com> <5568B352.8060005@marino.st> <556ABB1C.5080607@marino.st> <556B213E.8030108@marino.st> <556B4BE9.1000902@marino.st> <556BD14D.5050507@lcwb.coop> <556C9AF2.7090704@lcwb.coop> <556CFED5.80002@lcwb.coop> <20150602113611.66de95acf8cdb242c6672605@elegosoft.com> <556DD73B.9050601@lcwb.coop> Message-ID: <20150602211952.17478aefac4b66e13de704a9@elegosoft.com> On Tue, 02 Jun 2015 11:18:03 -0500 "Rodney M. Bates" wrote: > > > On 06/02/2015 04:36 AM, Olaf Wagner wrote: > > On Mon, 01 Jun 2015 19:54:45 -0500 > > "Rodney M. Bates" wrote: > > > >> Here is a short-term proposal (i.e., without major reorganization) > >> for the do-cm3*.sh scripts: > >> > >> 1) 'build' only builds, as we seem to agree it should. > >> 2) a new option 'override' (and only 'override') causes an override build > >> 3) a new option 'partialship' ships, as each package is done, things that > >> will be needed to compile another package that does a quake import on the > >> just-built package (I think this means static library, if any, and .M3WEB), > >> but does not ship things that will be used to execute the just-built package > >> (I think this means executable or dynamic library). I'm not sure right off > >> hand which ship group things like interface source files, etc. belong in. > > > > I don't know how you will implement this and how it should fit into the > > M3 package concept, unless you ship to a completely different location, > > i.e. another package pool (but cm3 currently just supports one). > > > > In my opinion the package system relies on the invariant that a shipped > > (installed) package is completely consistent, i.e. the source code > > interfaces correspond to the intermediate code interfaces and to the > > binary equivalent of them (static and dynamic libraries). If I understand > > you correctly, you want to give up this consistency in favour of being > > able to do an easier bootstrap of the CM3 system. > > Yes, but we already must have a temporarily inconsistent state of installation > today. This is a somewhat different kind of inconsistency, but inconsistent, > nonetheless. > > I guess -partialship could do an additional final pass over the package list, > doing a full ship. The inconsistent state would only last during a single > user-requested step, which is about as short as it can get. My intent was > that one would immediately do a full ship after a partialship, although as > a separate command, thus restoring consistency. OK, I think I have a better idea of what you want to do now. > > But bootstrapping a complex system is never easy and usually requires some > > special or tricky steps. Ordinary use of the system, i.e. all other applications, > > that just build on the standard tools, don't need this kind of steps. > > > > And my proposal is a simple, short-term way to provide somewhat more flexible > options for just such special or tricky steps. It will work the same way as now, > if you don't use the new options, and would allow Jay to get rid of the hated > INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps. > > > If I could design (and implement) a better cm3 builder, I'd have one with > > multiple package pools (shipping destinations, locations of installed > > packages) and an integrated builder that knows all about the package > > dependencies (so that we haven't to do that in scripts). The two points > > are the only shortcomings in the cm3 build system in my opinion. > > The prjm tools of the ComPact sources was a step in this direction: > > it can handle multiple pools of packages and their dependencies, but > > is language independent, therefore too complex, and never got integrated > > into the compiler builder. > > > > Yes, absolutely. We need a greatly reworked build system for multiple packages. > For building within a package, cm3's existing build is really quite sophisticated. > I don't know of any other that detects the need for recompilation > on declaration-granularity. But it does very little with inter-package > problems -- just detection of link-time inconsistencies, with hopelessly > uninformative error messages. (I once set out to improve them, but it > required more info in the .mx and .m3x files, with more incompatible > changes, and I only got one message improved a little bit, before I gave > up for the time.) > > I agree, reworking it all is very desirable. It will take a lot of rework. > I think we all agree that some kind of multi-layer, multi-destination > scheme is needed. This will probably entail disentangling the source > and build directories from being interspersed in each package, or something > equally perturbing to the existing system. It isn't going to be a little > patch. I think my proposal will be small to implement and have no impact > on anything existing, if you don't use the new options. I'm not against your proposal as a first or intermediate step. We have to remain pragmatic and improve things incrementally. If people really use the old shell scripts, I'm not against adapting them. Jay may do all the same things in Python even faster, but I always found that was an additional unnecessary dependency. On the other hand, Python can probably be installed in a couple of minutes on any system these days. I also agree that we would all like a multi-level fully integrated build system for sets of M3 packages, but that will be a lot of work and I won't be able to contribute much to it. Those who do the work will decide about the design and its capabilities. > > As for the shell scripts, I'm surprised that they have survived so long; > > they just got created on the fly to support the building and publication > > of the CM3 system as open source when we got the sources from Critical > > Mass. They were never intended as a general user interface, but only > > as tools for the CM3 maintainers, and later got used in the Hudson CI. > > > > BTW, if I find some time and access to suitable machines again, I'd > > really like to set up a new CI system on Jenkins interfacing with the > > Github infrastruture. But I won't make any promises here. > > Yes, this would be very good. Priorities, priorities! :-(. It's not as improbable as me writing a new integrated build system though, as I've set up several complex CI systems for other projects. We'll se if I find the fime... Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Jun 3 03:58:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 01:58:10 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem Message-ID: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 04:51:45 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 02:51:45 +0000 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) Message-ID: Ideally: 1 32bit host could target 64bit target 2 32bit texts could be as large as the address space/OS allows. 3 ditto 64bit texts -- larger than 4GB 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. Currently only #4 is true. TEXTs for any target can only be about 500MB. To fix 2 and 3 requires m3front to use TInt more. This fixes #1 and likely breaks #4. Cutting the limit just a little. Can we special case somehow? Is there some construct we can use that has no stated limit, like 0..cnt - 1? Should/can we introduce one? jbook2:src jay$ git diff "../src/text/TextLiteral.i3"diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..4d16a44 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; CONST (* DIV BITSIZE should not be here! *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ otherwise we have: new source -> compiling TextLiteral.i3"../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large1 error encountered Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 08:17:49 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 08:17:49 +0200 Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>, , , , , , , , , , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , <556C254D.7070306@marino.st>, , <556C2962.3020505@marino.st>, Message-ID: <556E9C0D.5000403@marino.st> On 6/2/2015 09:11, Jay K wrote: > John, this and your other problems should all be fixed now. > > I was able to reproduce them mostly. > I didn't actually reproduce the cvsup problem, but I see in the log you > did set CM3, and I did test with that, and hit the cm3cg version > mismatch, and the problem is pretty clear -- my mistake. > > I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && > make-dist.py or skip right to make-dist.py. > There was an additional problem skipping right to make-dist.py that I > fixed, at least for Darwin. Hi Jay, I set up the VM last night and it completed the make-dist.py command. I have some questions / comments. comment 1: Using bin for config isn't a standard unix location. e.g. rather than /bin/config/AMD64_FREEBSD, it should be located at /etc/modula3/config/AMD64_FREEBSD (can it be configurable?) comment 2: Many of the binaries are might actually be example programs, e.g. cube, calculator, fisheye. Rather than /bin where they risk clashing with other programs, it might be advisable to stick them a /share/examples/modula3 (configurable?). comment 3: /www is not standard either. We would be this in /share/doc/modula3. Can it be configurable? Comments 1--3 might be related because www might have relative paths that need changing comment 4: I'd like a configurable option not to tar/gzip the products. In my case, I'm going to stage them manually so this is an unnecessary step. comment 5: the "min" distribution seems suitable to be packaged as a bootstrap question 6: In most use cases, other than intentionally recreating the bootstrap, I'm not going to need the "min" distribution. Is there any impact to building "all" only? In other words, does something else use "min" distribution? Thanks for the update! John From jay.krell at cornell.edu Wed Jun 3 09:18:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 07:18:41 +0000 Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) In-Reply-To: <556E9C0D.5000403@marino.st> References: <556700B0.8060500@marino.st>, , , , , <556B213E.8030108@marino.st>,,, , , , , , , , , ,,, , , <556B4BE9.1000902@marino.st>, , , , , ,,, , , , , ,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , ,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , ,,<556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, ,,, , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st> Message-ID: "Yes". 1) "config"'s user configurability is over-implied imho. It shouldn't all be considered config. It needs refactoring into bin and config. Possibly rewriting the stuff that has stood the test of a few year's time into native Modula-3. It used to be worse. Each file used to be standalone. I refactored -- all the "common" files. I think it is partly called config because if you look at the gcc source tree, it also has a config directory. https://github.com/modula3/cm3/tree/master/m3-sys/m3cc/gcc/gcc/config It should be called "target" perhaps. Heck, we could have bin//cm3cg, bin//cm3.cfg or bin/config.quake or such. Consider all the Perl and Python and sh you may or may not have in /usr/bin. Is it configurable? User editable? Maybe/kinda. Consider that /etc is often sh snippets. The line between code and data is somewhat blurry, and I am somewhat lazy. We could split the files into "less editable" and "more editable". The bulk of the text in the files is hard to understand and hard to edit. And then there is the question of updating. And when I make edits to "the defaults", which do I assume are carried with cm3, vs. which do I assume an update will leave alone? Basically, I'm lazy, and haven't perfectly evaluated or factored everything. It used to be worse. However, there is/was cminstall, which the user is/was meant to run which would edit small parts. So maybe those very parts should be broken out into separate files? There another problem though. Which parts of "config"/cminstall need to be rerun when something else changes? You know, imagine I install cm3 and then I install X Windows. Installing X Windows implies a need to reconfigure. The full needs of a configuration system are related to a packaging system. Debian I believe handles this all well, after years of development focusing on these sorts of things. And then we'd have to handle all the other packaging systems. And nobody has been interested/motivated enough so far to really integrate so well into all the various packaging systems. But also, we target not just Unix. Yet we have almost the same directory structure on Unix and Windows. The difference is that on Windows we put the .dlls in bin. Should we use $HOME/etc or $HOME/config? And have /usr/local/cm3/bin probe around? I think I did put in some probing. To what extend do users just have private installs $HOME/cm3/bin vs. sharing binaries and having just split off config? What if they want to change the compiled code? To what extent are systems multi-user? So "yes" there is work to do here and we need help :) 2) Yes. Remember how I said I was lazy and made just one, or rather two, package sets? That is what you are seeing here. Olaf did put more thought/work into this and the .sh files do build a few different distribution sets. 3) I think so. Look for "www" in config? 1-3) yes. In particular, are you aware of the rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? I went with origin. I didn't have the energy or perhaps the buy-in to implement relink-upon-install approach. Previously we relied on either paths being correct *before* make install, or setting LD_LIBRARY_PATH. Both are bad solutions. There are I believe approx. 4 choices, for redistributable binaries that might go to a variety of setups: a) paths correct on original build machine b) LD_LIBARY_PATH c) origin d) relink upon install (leaving binaries not further relocatable by mv/cp). I went with c. So if you move samples around, you have to deal with that. bin/samples would seem easy enough though. Or heck, a peer of bin? lib? samplebin? samples? I think if we adopted automake/libtool, we'd get #c and it would be kinda good. Do cmake users use libtool? You know meta meta meta point -- we have our own little close to state of the art build system, but not quite state of the art. It works prety well and pretty portably. But not always as well as people would like. Between autoconf and Perl's method of port to every system, we are more like Perl, but also trying to find least common denominators that don't actually have to be ported. Does this make some sense? 4) Just command line parsing in make-dist.py. 5) yes 6) I don't understand. min is meant to be: 6a) a minimal toolset You can write hello world. cm3, cm3cg, mklib, config, m3core, libm3 text, threads not a gui This is subjective. The "resource" stuff should probably be here. 6b) It is meant to be enough to bootstrap another cm3 version from, older or newer. cm3 manages its dependencies carefully. It either carries things itself, or they are in min. For example, it doesn't use X Windows (or .i3 files describing it). upgrade.py/upgrade.sh assume min Now, granted, we can bootstrap from nothing as well. 7) You didn't ask. But I kinda think we should only have environment variables that start CM3 -- CM3 and start CM3_ actually. 8) Another large meta point -- this is all clearly imperfect, usually for lack of time or energy, not because we felt it was better this way. Though there is the angle I gave -- portability. Also, autoconf was probably less well established when most of this was written. The build/conf world seems still to be somewhat unsettled. 9) I have a larger unimplemented vision -- the system should be redistributed as one portable hard to read C or C++ .tar.gz. This doesn't work today, because "the frontend does layout" -- we essentially have 6 flavors of C: 1 posix 32bit little endian (x86, etc.) 2 posix 32bit big endian (ppc_darwin, sparc32) 3 posix 64bit little endian (amd64. etc.) 4 posix 64bit big endian (ppc64_darwin, sparc64) 5 Windows 32bit little endian (x86 mips PowerPC alpha, arm32, etc. ) 6 Windows 32bit big endian (hypothetical xbox 360? CE?) 7 Windows 64bit little endian (amd64, ia64, alpha64 etc.) 8 Windows 64bit big endian (hypothetical xbox 360?) We could today make 6 distributions of just C and bootstrap from one of them, picking the right one. Most 64bit systems can run 32bit, so strike those as bootstrap? But not quite, e.g. OpenBSD. I'd like the option for the frontend to defer layout to the backend, and for the C backend to output string expressions involving sizeof(size_t) or sizeof(char*). And, for win32 vs. posix, I'd like quake to compile both, and to generate a Makefile that picks the right one, or a .sh/.cmd. Oh, and jmpbuf size is in the generated C. I know how to fix that. bootstrap specifically could inflate it to the largest known case, which is quite large. Or, once we generate C++ for excpetion handling, jmpbuf should go away. - Jay > Date: Wed, 3 Jun 2015 08:17:49 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: [M3devel] It works! (was: m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110) > > On 6/2/2015 09:11, Jay K wrote: > > John, this and your other problems should all be fixed now. > > > > I was able to reproduce them mostly. > > I didn't actually reproduce the cvsup problem, but I see in the log you > > did set CM3, and I did test with that, and hit the cm3cg version > > mismatch, and the problem is pretty clear -- my mistake. > > > > I am able to start with I386_DARWIN 5.8.6 and either upgrade.py && > > make-dist.py or skip right to make-dist.py. > > There was an additional problem skipping right to make-dist.py that I > > fixed, at least for Darwin. > > Hi Jay, > I set up the VM last night and it completed the make-dist.py command. I > have some questions / comments. > > comment 1: > Using bin for config isn't a standard unix location. > e.g. rather than /bin/config/AMD64_FREEBSD, it should be located > at /etc/modula3/config/AMD64_FREEBSD (can it be configurable?) > > comment 2: > Many of the binaries are might actually be example programs, e.g. cube, > calculator, fisheye. Rather than /bin where they risk clashing > with other programs, it might be advisable to stick them a > /share/examples/modula3 (configurable?). > > comment 3: /www is not standard either. We would be this in > /share/doc/modula3. Can it be configurable? > > Comments 1--3 might be related because www might have relative paths > that need changing > > comment 4: I'd like a configurable option not to tar/gzip the products. > In my case, I'm going to stage them manually so this is an unnecessary > step. > > comment 5: the "min" distribution seems suitable to be packaged as a > bootstrap > > question 6: In most use cases, other than intentionally recreating the > bootstrap, I'm not going to need the "min" distribution. Is there any > impact to building "all" only? In other words, does something else use > "min" distribution? > > Thanks for the update! > John > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 10:36:51 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 10:36:51 +0200 Subject: [M3devel] It works! In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , <556B4BE9.1000902@marino.st>, , , , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , <556C16E7.505@marino.st>, , , , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st> Message-ID: <556EBCA3.3090601@marino.st> On 6/3/2015 09:18, Jay K wrote: > 1) "config"'s user configurability is over-implied imho. > [snip] > But also, we target not just Unix. > Yet we have almost the same directory structure on Unix and Windows. > The difference is that on Windows we put the .dlls in bin. It is possible this was misunderstood. It's important to release that cm3 will be installed nominally at /usr/local/bin. On a normal unix system, there could be over 500 packages all installing there. Things like configuration files (.conf) are nominally installed at /usr/local/etc/. The config directory is analogous to these. I will do the following in any case: A) move config from bin to etc// B) edit cm3.cfg to point to new location All I was suggesting is that the script do this for me. It can still install at bin/ by default, but it would be nice if it could install elsewhere too. > Should we use $HOME/etc or $HOME/config? Never. This does not work for packaging. E.g. prebuild binary packages that are installed (no building required) > 2) "example" progs > Yes. Remember how I said I was lazy and made just one, or rather two, > package sets? That is what you are seeing here. > Olaf did put more thought/work into this and the .sh files do build a > few different distribution sets. I am fine with what is built. Remember above that cm3 is installed with 500 other packages. Any program that is not essential and essentially there as an example could (and my opinion) should be installed in a place other than bin/. I don't care if it's installed outside of PATH because it probably won't be called much and absolute path is fine in this case. > 3) I think so. Look for "www" in config? > 1-3) yes. > In particular, are you aware of the > rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? I just remember that the current cm3 in ports, which is split up like I suggest, had some broken links because I moved these things around. I can't remember what is actually broken; I'd have to check again. The point was you can't just move stuff around without changing links, so the install script would have to do this if it supported alternative locations. > 4) > Just command line parsing in make-dist.py. Right, but currently I have to patch make-dist.py to make it not tar/gzip. If the script supported the option, I could avoid that patch. > 5) yes > 6) I don't understand. > min is meant to be: > 6a) a minimal toolset > You can write hello world. > cm3, cm3cg, mklib, config, m3core, libm3 > text, threads > not a gui > This is subjective. The "resource" stuff should probably be here. The port is only going to install one distribution, and that will be the "all". The non-used one is just a waste of building; it will be thrown away. Ports framework repackages and stored in FreeBSD's binary package format. > 7) You didn't ask. But I kinda think we should only have environment > variables > that start CM3 -- CM3 and start CM3_ actually. that sounds logical but as long as they are defined it doesn't really matter to me. > 8) Another large meta point -- this is all clearly imperfect, usually > for lack of time or energy, not because we felt it was better this way. > Though there is the angle I gave -- portability. > Also, autoconf was probably less well established when most of this was > written. > The build/conf world seems still to be somewhat unsettled. if a python script is sufficient for port needs, this issue isn't important to me. So far it seems that it is. > 9) I have a larger unimplemented vision -- the system should be > redistributed as one portable hard to read C or C++ .tar.gz. > This doesn't work today, because "the frontend does layout" -- we > essentially have 6 flavors of C: > We could today make 6 distributions of just C and bootstrap from one of > them, picking the right one. > Most 64bit systems can run 32bit, so strike those as bootstrap? But not > quite, e.g. OpenBSD. DragonFly is also 64-bit only. > I'd like the option for the frontend to defer layout to the backend, and > for the C backend to output string expressions involving sizeof(size_t) > or sizeof(char*). > And, for win32 vs. posix, I'd like quake to compile both, and to > generate a Makefile that picks the right one, or a .sh/.cmd. > Oh, and jmpbuf size is in the generated C. I know how to fix that. > bootstrap specifically could inflate it to the largest known case, which > is quite large. > Or, once we generate C++ for excpetion handling, jmpbuf should go away. no comment. :) John From jay.krell at cornell.edu Wed Jun 3 11:30:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 09:30:44 +0000 Subject: [M3devel] It works! In-Reply-To: <556EBCA3.3090601@marino.st> References: <556700B0.8060500@marino.st>, , , , ,,<556B4BE9.1000902@marino.st>, , , , ,,,,, , , , ,,,,<70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , ,,,,<807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , ,,, , , , , , , , <556C1218.3000103@marino.st> , , , , , <556C16E7.505@marino.st>, , , , ,,,,<556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st>, , <556EBCA3.3090601@marino.st> Message-ID: I believe people typically install to /usr/local/cm3 instead of /usr/local.There is a philosophical problem here in terms of setup file systemlayout. //package and then symlinks from /usr/local/{bin,lib}and removal of a package is just removing its one directory recursivelyand cleaning up any orphaned links vs. files put directly into/usr/local/{bin,lib} and data maintained on the side as to which filecame from which package. The first way gives more leeway to install extra stuff but not linkit into $PATH etc. The second way I believe is more common.It is easy to put together by make DESTDIR=whatever, tar.gz whateverand install is just untar, and uninstall either isn't implemented,or the data on the side is the output of tar tf, I guess. I put it in /cm3 on my systems and $HOME/cm3 where I don'thave that access, or $HOME/cm3. for filesystemsshared across targets -- e.g. the opencsw machines share filesystemacross sparc32, sparc64, x86, amd64. But yes, I understand. Just because it is called "config", does not make it "config".It should be refactored.And will there be some expectation that the "binaries" can beupdated while leaving "config" alone? > currently I have to patch make-dist.py to make it not > tar/gzip. If the script supported the option, I could avoid that patch. Send your patches here? Or fork on github and send pull requests? Another "less-binary" bootstrap option is the C archives.It is text files, but they are all generated and hard to read.I have to get the C backend back to working. :( Since you are going into ports, with an expected install place,and are building it yourself, you might want to mess withrpath, see here: jbook2:config-no-install jay$ pwd/dev2/cm3/m3-sys/cminstall/src/config-no-install book2:config-no-install jay$ grep rpath *ALPHA_OSF: args += [ "-rpath", LIB_USE ]FreeBSD.common: & " -Wl,-rpath,\\$ORIGIN"FreeBSD.common: & " -Wl,-rpath,\\$ORIGIN/../lib "Interix.common: % & " -Wl,-rpath,\\$ORIGIN"Interix.common: % & " -Wl,-rpath,\\$ORIGIN/../lib"Linux.common: & " -Wl,-rpath,\\$ORIGIN"Linux.common: & " -Wl,-rpath,\\$ORIGIN/../lib"NetBSD.common:M3_SHARED_LIB_ARG = "-Wl,-rpath-link,"OpenBSD.common:% & " -Wl,-rpath,\\$ORIGIN" % no $ORIGIN support up to/including 4.7OpenBSD.common:% & " -Wl,-rpath,\\$ORIGIN/../lib" % no $ORIGIN support up to/including 4.7OpenBSD.common:SYSTEM_LIBS{"ODBC"} = ["-Wl,-rpath,/usr/local/lib -L/usr/local/lib -liodbc" ]OpenBSD.common:SYSTEM_LIBS{"POSTGRES95"} = ["-Wl,-rpath -L/usr/local/lib -lpq"]gnuld.common: M3_SHARED_LIB_ARG = "-Wl,-rpath,"gnuld.common: GNU_LD_APPEND = " -Wl,-rpath," & INSTALL_ROOT & "/lib " % non-portablegnuld.common: M3_SHARED_LIB_ARG = "-Wl,-rpath," - Jay > Date: Wed, 3 Jun 2015 10:36:51 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] It works! > > On 6/3/2015 09:18, Jay K wrote: > > 1) "config"'s user configurability is over-implied imho. > > [snip] > > But also, we target not just Unix. > > Yet we have almost the same directory structure on Unix and Windows. > > The difference is that on Windows we put the .dlls in bin. > > It is possible this was misunderstood. > > It's important to release that cm3 will be installed nominally at > /usr/local/bin. On a normal unix system, there could be over 500 > packages all installing there. Things like configuration files (.conf) > are nominally installed at /usr/local/etc/. The config directory is > analogous to these. > > I will do the following in any case: > A) move config from bin to etc// > B) edit cm3.cfg to point to new location > > All I was suggesting is that the script do this for me. It can still > install at bin/ by default, but it would be nice if it could install > elsewhere too. > > > > > Should we use $HOME/etc or $HOME/config? > > Never. > This does not work for packaging. > E.g. prebuild binary packages that are installed (no building required) > > > > > 2) "example" progs > > Yes. Remember how I said I was lazy and made just one, or rather two, > > package sets? That is what you are seeing here. > > Olaf did put more thought/work into this and the .sh files do build a > > few different distribution sets. > > I am fine with what is built. Remember above that cm3 is installed with > 500 other packages. Any program that is not essential and essentially > there as an example could (and my opinion) should be installed in a > place other than bin/. I don't care if it's installed outside of PATH > because it probably won't be called much and absolute path is fine in > this case. > > > 3) I think so. Look for "www" in config? > > 1-3) yes. > > In particular, are you aware of the > > rpath/origin/relink-upon-make-install/LD_LIBRARY_PATH situation? > > > I just remember that the current cm3 in ports, which is split up like I > suggest, had some broken links because I moved these things around. I > can't remember what is actually broken; I'd have to check again. The > point was you can't just move stuff around without changing links, so > the install script would have to do this if it supported alternative > locations. > > > > 4) > > Just command line parsing in make-dist.py. > > Right, but currently I have to patch make-dist.py to make it not > tar/gzip. If the script supported the option, I could avoid that patch. > > > > 5) yes > > 6) I don't understand. > > min is meant to be: > > 6a) a minimal toolset > > You can write hello world. > > cm3, cm3cg, mklib, config, m3core, libm3 > > text, threads > > not a gui > > This is subjective. The "resource" stuff should probably be here. > > > The port is only going to install one distribution, and that will be the > "all". The non-used one is just a waste of building; it will be thrown > away. > > Ports framework repackages and stored in FreeBSD's binary package format. > > > 7) You didn't ask. But I kinda think we should only have environment > > variables > > that start CM3 -- CM3 and start CM3_ actually. > > that sounds logical but as long as they are defined it doesn't really > matter to me. > > > > 8) Another large meta point -- this is all clearly imperfect, usually > > for lack of time or energy, not because we felt it was better this way. > > Though there is the angle I gave -- portability. > > Also, autoconf was probably less well established when most of this was > > written. > > The build/conf world seems still to be somewhat unsettled. > > if a python script is sufficient for port needs, this issue isn't > important to me. So far it seems that it is. > > > 9) I have a larger unimplemented vision -- the system should be > > redistributed as one portable hard to read C or C++ .tar.gz. > > This doesn't work today, because "the frontend does layout" -- we > > essentially have 6 flavors of C: > > We could today make 6 distributions of just C and bootstrap from one of > > them, picking the right one. > > Most 64bit systems can run 32bit, so strike those as bootstrap? But not > > quite, e.g. OpenBSD. > > DragonFly is also 64-bit only. > > > I'd like the option for the frontend to defer layout to the backend, and > > for the C backend to output string expressions involving sizeof(size_t) > > or sizeof(char*). > > And, for win32 vs. posix, I'd like quake to compile both, and to > > generate a Makefile that picks the right one, or a .sh/.cmd. > > Oh, and jmpbuf size is in the generated C. I know how to fix that. > > bootstrap specifically could inflate it to the largest known case, which > > is quite large. > > Or, once we generate C++ for excpetion handling, jmpbuf should go away. > > no comment. :) > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 11:54:50 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 11:54:50 +0200 Subject: [M3devel] It works! In-Reply-To: References: <556700B0.8060500@marino.st>, , , , , , , , , , , , , , <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com>, , , , , , , , <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com>, , , , , , , , <556C04AD.5000509@marino.st>, <556C0765.5050607@marino.st>, , , , , , , , , , , , , , , , <556C1218.3000103@marino.st> , , , , , <556C16E7.505@marino.st>, , , , , , , , <556C1ED3.3000303@marino.st>, , <556C21AA.70501@marino.st>, , , , , , , , <556C254D.7070306@marino.st>, , , , <556C2962.3020505@marino.st>, , , , <556E9C0D.5000403@marino.st>, , <556EBCA3.3090601@marino.st> Message-ID: <556ECEEA.9040302@marino.st> On 6/3/2015 11:30, Jay K wrote: > I believe people typically install to /usr/local/cm3 instead of /usr/local. > There is a philosophical problem here in terms of setup file system > layout. //package and then symlinks from /usr/local/{bin,lib} > and removal of a package is just removing its one directory recursively > and cleaning up any orphaned links vs. files put directly into > /usr/local/{bin,lib} and data maintained on the side as to which file > came from which package. Well, setting it to /usr/local/cm3 would eliminate name clashes, but it would also move cm3 out of the standard path. That would require every user to adjust their path. If were were talking about 1-3 symlinks in /usr/local/bin then that would be fine, but if it's like 25+ then that could be kind of pointless. Plus on FreeBSD documentation, config, and examples go in standard locations (if possible) and setting to /usr/local/cm3 without modification wouldn't solve that. We aren't worried about orphan files or whatever, the binary package manager cleans up after itself when a package is removed, and won't let two versions of a package be installed simultaneously, so there's no real benefit to having cm3 in a dedicated directory (IMO). > Just because it is called "config", does not make it "config". > It should be refactored. > And will there be some expectation that the "binaries" can be > updated while leaving "config" alone? No. With updates, everything that was installed gets removed. If this is part of an update, everything is reinstalled. There's always integrity there. For BSD setups, having it at etc/modula3/config makes a lot of sense and there's no downside other than it's a different location from releases that you guys may make. > > currently I have to patch make-dist.py to make it not > > tar/gzip. If the script supported the option, I could avoid that patch. > > > Send your patches here? > Or fork on github and send pull requests? They wouldn't be suitable. They would be patches to make the script work meaning lines would be removed. You're thinking about a patch to handle both cases. > Since you are going into ports, with an expected install place, > and are building it yourself, you might want to mess with > rpath, see here: IIRC, the rpath was already fixed with the CM3 portable rpath option. I sort of recall hitting that before and finding $ORIGIN supported. If RPATH is wrong it definitely needs fixing but I think it is a problem that presents quickly (e.g. you couldn't use the product as a bootstrap). John From wagner at elegosoft.com Wed Jun 3 13:11:02 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 3 Jun 2015 13:11:02 +0200 Subject: [M3devel] It works! In-Reply-To: <556ECEEA.9040302@marino.st> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> Message-ID: <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> On Wed, 03 Jun 2015 11:54:50 +0200 John Marino wrote: > On 6/3/2015 11:30, Jay K wrote: > > I believe people typically install to /usr/local/cm3 instead of /usr/local. > > There is a philosophical problem here in terms of setup file system > > layout. //package and then symlinks from /usr/local/{bin,lib} > > and removal of a package is just removing its one directory recursively > > and cleaning up any orphaned links vs. files put directly into > > /usr/local/{bin,lib} and data maintained on the side as to which file > > came from which package. > > Well, setting it to /usr/local/cm3 would eliminate name clashes, but it > would also move cm3 out of the standard path. That would require every > user to adjust their path. > > If were were talking about 1-3 symlinks in /usr/local/bin then that > would be fine, but if it's like 25+ then that could be kind of pointless. > > Plus on FreeBSD documentation, config, and examples go in standard > locations (if possible) and setting to /usr/local/cm3 without > modification wouldn't solve that. > > We aren't worried about orphan files or whatever, the binary package > manager cleans up after itself when a package is removed, and won't let > two versions of a package be installed simultaneously, so there's no > real benefit to having cm3 in a dedicated directory (IMO). > > > Just because it is called "config", does not make it "config". > > It should be refactored. > > And will there be some expectation that the "binaries" can be > > updated while leaving "config" alone? > > No. With updates, everything that was installed gets removed. If this > is part of an update, everything is reinstalled. There's always > integrity there. For BSD setups, having it at etc/modula3/config makes > a lot of sense and there's no downside other than it's a different > location from releases that you guys may make. It doesn't really fit, but never mind. What is different about M3 is that it contains both the base development system and dozens of utility and application packages, and every user is supposed to be able to update and install packages (only it's called shipping, not installation). Theoretically, you could package each M3 package in its own BSD package; I even think there was one (historical) distribution of PM3 that did exactly that. That doesn't address the problem, as each user needs to become root to ship a package. As I've written in another m3 thread yesterday, what would really be needed is a multi-level package pool hierarchy and an integrated m3 builder that copes with that and sets of packages. But even if we had that, I'm not sure how it would fit into the standard Unix file system hierarchy layout: put the user-managed package pools under /var/local/lib? As the package systems vary significantly between all the supported platforms of M3, the concepts of M3 packages and OS packages have never been unified. And of course the M3 project never had enough personal resources to provide OS specific packages for even a small subset of the target platforms. Perhaps a pragmatical way to handle this would be to package and install a minimal system only containing the builder, compiler and base libraries as an OS package in an OS-specific way (including documentation and all) and add support for the standard M3 package hierarchy under /usr/local/cm3 (group-writable by an m3 group). It would be a compromise, but perhaps a better one than installing the complete CM3 distribution as one big package read-only for the ordinary user. The CM3 package pool under /usr/local/cm3 could even be integrated with the Github repository and allow easy checkout of all M3 packages there and even pushing new branches and packages. The builder could locate, checkout and install missing imported packages, have an option for creating a new package template and a standard way to publish this on Gibhub. This might help to lower the threshold for new users to become active in the community and share their code. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From adacore at marino.st Wed Jun 3 13:46:07 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 13:46:07 +0200 Subject: [M3devel] It works! In-Reply-To: <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> Message-ID: <556EE8FF.3060502@marino.st> On 6/3/2015 13:11, Olaf Wagner wrote: > What is different about M3 is that it contains both the base development > system and dozens of utility and application packages, and every user > is supposed to be able to update and install packages (only it's > called shipping, not installation). If I just install the "all" distribution, as built, in /usr/local/cm3 (with the possible exception of shifting www to standard docs location): 1) It is a read-only area (ideally) 2) It would require all users to set /usr/local/cm3 in PATH. This is not really that big of a deal. It's not the first package to require something like that, plus, frankly, the m3 community is not large and such a requirement could be imposed without a lot of backlash. Maybe this is the best solution after all (and again it makes the port makefile simpler) > Theoretically, you could package each M3 package in its own BSD > package; I even think there was one (historical) distribution of PM3 > that did exactly that. That doesn't address the problem, as each user > needs to become root to ship a package. Breaking each into it's own package is what should happen, but that requires work on my part that I'm not able to give, so everyone has to deal with the giant port. If m3 regains enough popularity to justify splitting into individual packages then we can revisit the decision. > As I've written in another m3 thread yesterday, what would really > be needed is a multi-level package pool hierarchy and an integrated > m3 builder that copes with that and sets of packages. But even if we had > that, I'm not sure how it would fit into the standard Unix file system > hierarchy layout: put the user-managed package pools under /var/local/lib? At least for ports, packages either go in root's areas (e.g. /usr/local) or user-accessible areas but not mixed. One can install M3 in a user's area and do what they want. But a generic user can't install to standard areas like /usr/local and most /var areas. > As the package systems vary significantly between all the supported > platforms of M3, the concepts of M3 packages and OS packages have > never been unified. And of course the M3 project never had enough > personal resources to provide OS specific packages for even a small > subset of the target platforms. Pragmatically, you want the individual platforms to package it for you. Less work for you and it's probably done more correctly. E.g. m3 is self-hosting on FreeBSD so there's no real need to provide FreeBSD support now (with the possible exception of a statically linked "min" distribution) Ideally, similar support occurs on Debian, Fedora, Arch, macports, etc. John From adacore at marino.st Wed Jun 3 13:57:38 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 13:57:38 +0200 Subject: [M3devel] Are all 5 gcc branches used? Message-ID: <556EEBB2.30809@marino.st> I see 5 gcc branches in m3-sys/m3cc. I've seen gcc-4.7 in use and I can guess gcc-apple is still used, but are gcc, gcc-4.5, and gcc-4.6 branches obsolete? There reason why it matters is that I'm using github's built-in "create tarball from top of repo" capability as the digest-confirmed source distribution file. All the extra gcc branches are causing the compressed tarball to be pretty big. Currently it's at 150Mb. If some of these gcc branches are unused and will never again be used, I might suggest they are pruned. It's in the repository so they could be retrieved with the right hash, and it would dramatically reduce the size of the archive tarball. If all 5 versions of gcc are still used then that's a horse of a different color. John From jay.krell at cornell.edu Wed Jun 3 18:36:17 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 3 Jun 2015 09:36:17 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: > Other than cross-compile from 32 to 64 bit what purpose does this serve? > > Sent from my iPad > > On Jun 2, 2015, at 9:58 PM, Jay K wrote: > >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 >> index fa72589..37bf238 100644 >> --- a/m3-libs/m3core/src/text/TextLiteral.i3 >> +++ b/m3-libs/m3core/src/text/TextLiteral.i3 >> @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; >> IMPORT RTHooks, TextClass; >> >> CONST >> - (* DIV BITSIZE should not be here! *) >> +(* When cm3 uses TInt and when pickle special cases this, it should be approx: >> + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) >> +(* This fails for 32bit host and 64bit target: >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) >> +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) >> + dependent. *) >> + (* DIV BITSIZE should not be here -- compiler limitation due to >> + non-use of TInt. *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; >> (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) >> (* Do not adjust this for INTEGER size. It makes T have different >> fingerprints on 32- and 64-bit machines, which undermines pickling/ >> >> >> Yes I know TEXT pickles will be broken. >> I propose that unpickle should special case a few values here. >> Or, isn't the brand supposed to help? >> Or, can we encode this in a better way, w/o an upper bound? >> I know if you put in 0..0 or 0..-1 you incur range violations at runtime. >> Which makes me wonder -- are TEXTs unsafe? >> >> Should we support a syntax something like: >> >> cnt : INTEGER; >> buf : ARRAY [0..cnt - 1] OF Byte; >> ? >> >> Thanks, >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 18:39:46 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 3 Jun 2015 09:39:46 -0700 Subject: [M3devel] It works! In-Reply-To: <556EE8FF.3060502@marino.st> References: <556700B0.8060500@marino.st> <70797A4C-C2D5-4098-BC5E-DCBFA7287949@gmail.com> <807FEA8A-DE38-4128-8AE9-1ED0FC73AE67@gmail.com> <556C04AD.5000509@marino.st> <556C0765.5050607@marino.st> <556C1218.3000103@marino.st> <556C16E7.505@marino.st> <556C1ED3.3000303@marino.st> <556C21AA.70501@marino.st> <556C254D.7070306@marino.st> <556C2962.3020505@marino.st> <556E9C0D.5000403@marino.st> <556EBCA3.3090601@marino.st> <556ECEEA.9040302@marino.st> <20150603131102.87c46d85c8c6dbeeee6fbe0a@elegosoft.com> <556EE8FF.3060502@marino.st> Message-ID: <5649AA84-7127-49E1-9F80-7E3F425A2DE5@gmail.com> I am sympathetic to "path pollution" concerns even if in usr/local/cm3. We can move some stuff to demobin or samplebin if agreement on uselessness. And/or prefix every bin & lib with m3 or cm3. We should also rev our so versions? & and install version-named cm3 & cm3cg? - Jay On Jun 3, 2015, at 4:46 AM, John Marino wrote: > On 6/3/2015 13:11, Olaf Wagner wrote: >> What is different about M3 is that it contains both the base development >> system and dozens of utility and application packages, and every user >> is supposed to be able to update and install packages (only it's >> called shipping, not installation). > > If I just install the "all" distribution, as built, in /usr/local/cm3 > (with the possible exception of shifting www to standard docs location): > > 1) It is a read-only area (ideally) > 2) It would require all users to set /usr/local/cm3 in PATH. > > This is not really that big of a deal. It's not the first package to > require something like that, plus, frankly, the m3 community is not > large and such a requirement could be imposed without a lot of backlash. > > Maybe this is the best solution after all (and again it makes the port > makefile simpler) > > >> Theoretically, you could package each M3 package in its own BSD >> package; I even think there was one (historical) distribution of PM3 >> that did exactly that. That doesn't address the problem, as each user >> needs to become root to ship a package. > > Breaking each into it's own package is what should happen, but that > requires work on my part that I'm not able to give, so everyone has to > deal with the giant port. If m3 regains enough popularity to justify > splitting into individual packages then we can revisit the decision. > >> As I've written in another m3 thread yesterday, what would really >> be needed is a multi-level package pool hierarchy and an integrated >> m3 builder that copes with that and sets of packages. But even if we had >> that, I'm not sure how it would fit into the standard Unix file system >> hierarchy layout: put the user-managed package pools under /var/local/lib? > > At least for ports, packages either go in root's areas (e.g. /usr/local) > or user-accessible areas but not mixed. One can install M3 in a user's > area and do what they want. But a generic user can't install to > standard areas like /usr/local and most /var areas. > > >> As the package systems vary significantly between all the supported >> platforms of M3, the concepts of M3 packages and OS packages have >> never been unified. And of course the M3 project never had enough >> personal resources to provide OS specific packages for even a small >> subset of the target platforms. > > Pragmatically, you want the individual platforms to package it for you. > Less work for you and it's probably done more correctly. E.g. m3 is > self-hosting on FreeBSD so there's no real need to provide FreeBSD > support now (with the possible exception of a statically linked "min" > distribution) > > Ideally, similar support occurs on Debian, Fedora, Arch, macports, etc. > > John > > From jay.krell at cornell.edu Wed Jun 3 19:49:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 17:49:07 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> Message-ID: I don't have any, but should we have such artificial limits? I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." And even that isn't right -- 4bg is often not lenty for a 32bit host. File sizes exceed that long ago.But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. Given that a blueray movie is an array of bytes, is a text inappropriate? The actual diff presented just adjusts the limit by 8 bytes. Ok? Or must preserve pickles and can't do this w/o some other change? This isn't a more general problem? Or usually pickles are 32/64-compatible, but this is a most unusual case? Thanks, - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 13:24:36 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Why would you ever want a TEXT literal that is so large?By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface.A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. On Jun 3, 2015, at 12:36 PM, Jay wrote:I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: Other than cross-compile from 32 to 64 bit what purpose does this serve? Sent from my iPad On Jun 2, 2015, at 9:58 PM, Jay K wrote: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 3 20:04:04 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 18:04:04 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556EEBB2.30809@marino.st> References: <556EEBB2.30809@marino.st> Message-ID: They likely aren't all used.I'd like to make none of them used, but that is another matter. Look in src/m3makefile. I think OpenBSD targets might be back on gcc-4.2.There is flexibility in there, in m3makefile and "parse.c", but maybe at too high cost. The problem, you can imagine, is that Apple has as fork, OpenBSD has a fork, we have a fork, and there is mainline.Ideally everyone would be in mainline and nobody would have any forks. As we/I move from major version to version, I generally make a new fork. I have a certain paranoia and laziness when debugging -- "does it work with the old one" -- "how easy is to reconstruct the old one? Oh, cool, it is still right there, I don't have to figure out how to go backwards in source control, and I can easily have them both side by side". I realize source control could help me here. It also allows for "staging", i.e. I can bring in new version, test some targets, but leave the other targets alone, waiting for myself or others to test them later. Or, decide they are little enough used just move them forward w/o testing. If gcc obsoletes targets we want to keep, we could keep the old versions. Not super useful given our usage levels. We could maybe also minimize our changes and distribute patches only.Sometimes maybe I went overboard with my changes. Or we could ditch gcc entirely and use the C backend and/or LLVM, hopefully both unpatched. :) Try xz instead of gzip, maybe it halves the size? - Jay > Date: Wed, 3 Jun 2015 13:57:38 +0200 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Are all 5 gcc branches used? > > I see 5 gcc branches in m3-sys/m3cc. I've seen gcc-4.7 in use and I can > guess gcc-apple is still used, but are gcc, gcc-4.5, and gcc-4.6 > branches obsolete? > > There reason why it matters is that I'm using github's built-in "create > tarball from top of repo" capability as the digest-confirmed source > distribution file. All the extra gcc branches are causing the > compressed tarball to be pretty big. Currently it's at 150Mb. > > If some of these gcc branches are unused and will never again be used, I > might suggest they are pruned. It's in the repository so they could be > retrieved with the right hash, and it would dramatically reduce the size > of the archive tarball. > > If all 5 versions of gcc are still used then that's a horse of a > different color. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Wed Jun 3 20:10:07 2015 From: adacore at marino.st (John Marino) Date: Wed, 03 Jun 2015 20:10:07 +0200 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st> Message-ID: <556F42FF.9000106@marino.st> On 6/3/2015 20:04, Jay K wrote: > If gcc obsoletes targets we want to keep, we could keep the old > versions. Not super useful given our usage levels. It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not obsolete by that definition, so all 5 of these are obsolete. I would definitely encourage to prune as many of these as possible. I could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" based on the base compiler, why are you assuming it must be built by base compiler? By definition, the base compiler is only required to build base. There are much newer and well maintained versions of gcc in openbsd ports tree. There's no reason a ports compiler couldn't be used (I assume this is actually common). > Try xz instead of gzip, maybe it halves the size? I can't influence github's API. It is what it is. John From hendrik at topoi.pooq.com Wed Jun 3 21:07:46 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 3 Jun 2015 15:07:46 -0400 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st> Message-ID: <20150603190745.GA11809@topoi.pooq.com> On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > hopefully both unpatched. :) Weren't you working on a C backend for Modula a year or two ago? What happened to it? Is it available for use? If so, where and how? -- hendrik From rodney_bates at lcwb.coop Wed Jun 3 21:23:07 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 03 Jun 2015 14:23:07 -0500 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: References: Message-ID: <556F541B.5080207@lcwb.coop> On 06/02/2015 09:51 PM, Jay K wrote: > Ideally: > 1 32bit host could target 64bit target > 2 32bit texts could be as large as the address space/OS allows. > 3 ditto 64bit texts -- larger than 4GB > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > Currently only #4 is true. > TEXTs for any target can only be about 500MB. > > > To fix 2 and 3 requires m3front to use TInt more. > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > Can we special case somehow? > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > Should/can we introduce one? > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > index fa72589..4d16a44 100644 > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > CONST > (* DIV BITSIZE should not be here! *) > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; I'd say go ahead and change this. We really do want it to cross-compile sooner than anybody will be able to put TInt in all over the place. I did not build a cross compiler, but did what I think should be a realistic simulation of it by changing the cnt:INTEGER field to two INTEGER fields and compiling with a 32-bit compiler. That would be the same size as a 64-bit INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to compile, with bit size having space for 31 more bits before it overflows. This byte count of buf is a multiple of 4, which we want for unicode-range WIDECHAR. Making it -10 overflows, presumably because the additional byte is padded out to a multiple of 32 bits. I would have expected that somewhere, a size including the one-word heap header would be computed, but these are never actually heap allocated, and it seems to work. But to hedge against things, I suggest using -19. This would allow space to count it, cross-compiling 32->64. This will allow a text literal of 268435436 ISO characters, and a wide text literal of 1/4 that, which is not likely to be a serious limitation, considering that it all has to be on a single line of source code. As for pickles, the horse is already out of the barn on that, as pickles could already have been written with the current fingerprint, and we can't change the type in any way without the fingerprint changing. It is not hard to add recognition of the old fingerprint to pickles. Right off hand, I don't remember the process for finding actual fingerprint values, but I've done it before and can rediscover it. > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > (* Do not adjust this for INTEGER size. It makes T have different > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > otherwise we have: > > > new source -> compiling TextLiteral.i3 > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > 1 error encountered > > > Ok? > - Jay -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Jun 3 21:29:24 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 03 Jun 2015 14:29:24 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <556F5594.7070701@lcwb.coop> On 06/02/2015 08:58 PM, Jay K wrote: > We cannot cross from 32bit host to 64bit target. > > > Ok to commit this? > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > index fa72589..37bf238 100644 > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; > IMPORT RTHooks, TextClass; > CONST > - (* DIV BITSIZE should not be here! *) > +(* When cm3 uses TInt and when pickle special cases this, it should be approx: > + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) > +(* This fails for 32bit host and 64bit target: > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) > +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) > + dependent. *) > + (* DIV BITSIZE should not be here -- compiler limitation due to > + non-use of TInt. *) > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > (* Do not adjust this for INTEGER size. It makes T have different > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > Yes I know TEXT pickles will be broken. > I propose that unpickle should special case a few values here. > Or, isn't the brand supposed to help? > Or, can we encode this in a better way, w/o an upper bound? > I know if you put in 0..0 or 0..-1 you incur range violations at runtime. > Which makes me wonder -- are TEXTs unsafe? > > Should we support a syntax something like: > > cnt : INTEGER; > buf : ARRAY [0..cnt - 1] OF Byte; > ? > Can't do this. Non-open arrays must have static constant bounds, which the above does not. Open arrays have extra dope and are accessed through two extra pointers, which we really wouldn't want for text literals. TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that anything else in UNSAFE code. The interface is UNSAFE, which means safe code can't IMPORT it, and thus can't see the declaration. Not very likely that anybody other than the runtime, compiler-generated code, and debugger (written in C anyway) would want to mess with it. > Thanks, > - Jay -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 4 00:55:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 22:55:00 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556F42FF.9000106@marino.st> References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: > obsolete I meant, gcc 4.9 might not target Alpha/OSF, but gcc 4.5 might. > OpenBSD gcc 4.2. True -- we could take the OpenBSD port of gcc 4.7 or such and apply those patches. In fact... 1) most of the patches are to the driver, that we don't use Things like building with #define __OpenBSD__ and making size_t == unsigned long or unsigned int. 2) Maybe I'm doing this. I recall checking patch files and applying them either at build time or commiting those. 3) Really, you know, from a compiler backend point of view, OpenBSD, NetBSD, FreeBSD, all the same thing -- same ABI, same x86 instructions, same object file format. Various targets collapse down to fewer. So we don't always need the patches. But then something like ARM_DARWIN, which I never got fully working, those might be some more serious patches. 4.2 might also be Apple or Interix related. I'll have to look. We could probably also getting away with such things as: OpenBSD, Interix, ARM_DARWIN: C backend only, if even that I do have to restore it to working. It was totally working. Anyway, point taken/reminded -- we should see about pruning. I just get nervous, you know, I'm not a CVS or git expert -- how do I get back the old stuff? > xz vs. gzip Does the ports system now download from github? Cool. gzip is much faster for compression. - Jay > Date: Wed, 3 Jun 2015 20:10:07 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On 6/3/2015 20:04, Jay K wrote: > > If gcc obsoletes targets we want to keep, we could keep the old > > versions. Not super useful given our usage levels. > > It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not > obsolete by that definition, so all 5 of these are obsolete. > > I would definitely encourage to prune as many of these as possible. I > could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" > based on the base compiler, why are you assuming it must be built by > base compiler? > > By definition, the base compiler is only required to build base. There > are much newer and well maintained versions of gcc in openbsd ports > tree. There's no reason a ports compiler couldn't be used (I assume > this is actually common). > > > Try xz instead of gzip, maybe it halves the size? > > I can't influence github's API. It is what it is. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 00:58:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 22:58:50 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <20150603190745.GA11809@topoi.pooq.com> References: <556EEBB2.30809@marino.st>, , <20150603190745.GA11809@topoi.pooq.com> Message-ID: It was totally working. I built the whole Darwin x86/amd64 system, ran gui apps.Got AMD64_NT working.Built all four variations of Solaris I think.Maybe cross bootstrapped but didn't c_compile other systems.I386_NT was probably working. And now if you try it there are assertion failures, like about right shiftinga signed type. Some errors about duplicate typedefs. It has bit-rotted slightly. Maybe with the Unicode work? Anyway, I'll repair it. It really was fully working.Debugging was getting better, better than non-m3gdb systems.There is still a uniquifier appended to every local.In case you have; int a;{ int a;{ int a;}}} we give youint a_1;int a_2;int a_3; I need to fix it to only uniqueify in the presence of duplicates. Globals should be made debuggable instead of just being an array of bytes.Other than globals, structs have fields.TEXTs aren't viewable I think. But for systems like Darwin and NT and HP-UX that don't support m3gdb, debugging is already better.Build times are noticeably regressed. - Jay > Date: Wed, 3 Jun 2015 15:07:46 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > > hopefully both unpatched. :) > > Weren't you working on a C backend for Modula a year or two ago? > What happened to it? Is it available for use? If so, where and how? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:04:43 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:04:43 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st>, , , , <20150603190745.GA11809@topoi.pooq.com>, Message-ID: Once it is working, the way you use it is one of: ./do-cm3-all.py c buildship or change M3_BACKEND_MODE to "C" in the config file. and it handles nested functions even... You can bootstrap with./boot1.py c which gives you a directory of c files and a makefile.cd there and make or nmake. But let me fix it first. It got broken recently. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Wed, 3 Jun 2015 22:58:50 +0000 Subject: Re: [M3devel] Are all 5 gcc branches used? It was totally working. I built the whole Darwin x86/amd64 system, ran gui apps.Got AMD64_NT working.Built all four variations of Solaris I think.Maybe cross bootstrapped but didn't c_compile other systems.I386_NT was probably working. And now if you try it there are assertion failures, like about right shiftinga signed type. Some errors about duplicate typedefs. It has bit-rotted slightly. Maybe with the Unicode work? Anyway, I'll repair it. It really was fully working.Debugging was getting better, better than non-m3gdb systems.There is still a uniquifier appended to every local.In case you have; int a;{ int a;{ int a;}}} we give youint a_1;int a_2;int a_3; I need to fix it to only uniqueify in the presence of duplicates. Globals should be made debuggable instead of just being an array of bytes.Other than globals, structs have fields.TEXTs aren't viewable I think. But for systems like Darwin and NT and HP-UX that don't support m3gdb, debugging is already better.Build times are noticeably regressed. - Jay > Date: Wed, 3 Jun 2015 15:07:46 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On Wed, Jun 03, 2015 at 06:04:04PM +0000, Jay K wrote: > > > > Or we could ditch gcc entirely and use the C backend and/or LLVM, > > hopefully both unpatched. :) > > Weren't you working on a C backend for Modula a year or two ago? > What happened to it? Is it available for use? If so, where and how? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:14:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:14:49 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> Message-ID: sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched.An array of chars? Probably has same problem. Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 14:07:02 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu It?s only TEXT literals that are limited here. As in, what appears in a source program. On Jun 3, 2015, at 1:49 PM, Jay K wrote:I don't have any, but should we have such artificial limits? I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." And even that isn't right -- 4bg is often not lenty for a 32bit host. File sizes exceed that long ago.But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. Given that a blueray movie is an array of bytes, is a text inappropriate? The actual diff presented just adjusts the limit by 8 bytes. Ok? Or must preserve pickles and can't do this w/o some other change? This isn't a more general problem? Or usually pickles are 32/64-compatible, but this is a most unusual case? Thanks, - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Wed, 3 Jun 2015 13:24:36 -0400 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Why would you ever want a TEXT literal that is so large?By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface.A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. On Jun 3, 2015, at 12:36 PM, Jay wrote:I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. - Jay On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: Other than cross-compile from 32 to 64 bit what purpose does this serve? Sent from my iPad On Jun 2, 2015, at 9:58 PM, Jay K wrote: We cannot cross from 32bit host to 64bit target. Ok to commit this? diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3index fa72589..37bf238 100644--- a/m3-libs/m3core/src/text/TextLiteral.i3+++ b/m3-libs/m3core/src/text/TextLiteral.i3@@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; IMPORT RTHooks, TextClass; CONST- (* DIV BITSIZE should not be here! *)+(* When cm3 uses TInt and when pickle special cases this, it should be approx:+ MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *)+(* This fails for 32bit host and 64bit target:+ MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *)+(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER)+ dependent. *)+ (* DIV BITSIZE should not be here -- compiler limitation due to+ non-use of TInt. *) (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *)- MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) (* Do not adjust this for INTEGER size. It makes T have different fingerprints on 32- and 64-bit machines, which undermines pickling/ Yes I know TEXT pickles will be broken.I propose that unpickle should special case a few values here.Or, isn't the brand supposed to help?Or, can we encode this in a better way, w/o an upper bound?I know if you put in 0..0 or 0..-1 you incur range violations at runtime.Which makes me wonder -- are TEXTs unsafe? Should we support a syntax something like: cnt : INTEGER; buf : ARRAY [0..cnt - 1] OF Byte;? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 01:18:11 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 3 Jun 2015 23:18:11 +0000 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: <556F541B.5080207@lcwb.coop> References: , <556F541B.5080207@lcwb.coop> Message-ID: Should we just be a little lax and say minus 64?Or pick some other number like 256MB? Can we declare there is no bound? Thank you, - Jay > Date: Wed, 3 Jun 2015 14:23:07 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) > > > > On 06/02/2015 09:51 PM, Jay K wrote: > > Ideally: > > 1 32bit host could target 64bit target > > 2 32bit texts could be as large as the address space/OS allows. > > 3 ditto 64bit texts -- larger than 4GB > > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > > Currently only #4 is true. > > TEXTs for any target can only be about 500MB. > > > > > > To fix 2 and 3 requires m3front to use TInt more. > > > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > > Can we special case somehow? > > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > > Should/can we introduce one? > > > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > > index fa72589..4d16a44 100644 > > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > > CONST > > (* DIV BITSIZE should not be here! *) > > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > > I'd say go ahead and change this. We really do want it to cross-compile > sooner than anybody will be able to put TInt in all over the place. > > I did not build a cross compiler, but did what I think should be a realistic > simulation of it by changing the cnt:INTEGER field to two INTEGER fields and > compiling with a 32-bit compiler. That would be the same size as a 64-bit > INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to > compile, with bit size having space for 31 more bits before it overflows. > This byte count of buf is a multiple of 4, which we want for unicode-range > WIDECHAR. Making it -10 overflows, presumably because the additional byte is > padded out to a multiple of 32 bits. > > I would have expected that somewhere, a size including the one-word heap header > would be computed, but these are never actually heap allocated, and it seems to > work. But to hedge against things, I suggest using -19. This would allow space > to count it, cross-compiling 32->64. > > This will allow a text literal of 268435436 ISO characters, and a wide text literal > of 1/4 that, which is not likely to be a serious limitation, considering that it > all has to be on a single line of source code. > > As for pickles, the horse is already out of the barn on that, as pickles could > already have been written with the current fingerprint, and we can't change the > type in any way without the fingerprint changing. It is not hard to add > recognition of the old fingerprint to pickles. Right off hand, I don't remember > the process for finding actual fingerprint values, but I've done it before and > can rediscover it. > > > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > > (* Do not adjust this for INTEGER size. It makes T have different > > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > > > > > otherwise we have: > > > > > > new source -> compiling TextLiteral.i3 > > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > > 1 error encountered > > > > > > Ok? > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Thu Jun 4 16:01:37 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Thu, 04 Jun 2015 16:01:37 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: Message-ID: <55705A41.7070503@elstel.org> Am 03.06.15 um 03:58 schrieb Jay K: > We cannot cross from 32bit host to 64bit target. > > > Ok to commit this? > > Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. To me this is just another argument for a language specified value range of INTEGER. It makes certain things easier. My suggestion was: TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) BITS 64 FOR INTEGER would only work on x64 systems. The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same tends to be much slower). Regards, Elmar > Can't do this. Non-open arrays must have static constant bounds, > which the above does not. Open arrays have extra dope and are accessed > through two extra pointers, which we really wouldn't want for text > literals. > > TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that > anything > else in UNSAFE code. The interface is UNSAFE, which means safe code > can't IMPORT > it, and thus can't see the declaration. Not very likely that anybody > other than > the runtime, compiler-generated code, and debugger (written in C > anyway) would > want to mess with it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 4 16:07:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 09:07:48 -0500 Subject: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) In-Reply-To: References: , <556F541B.5080207@lcwb.coop> Message-ID: <55705BB4.4090408@lcwb.coop> On 06/03/2015 06:18 PM, Jay K wrote: > Should we just be a little lax and say minus 64? > Or pick some other number like 256MB? > Sure, just to allow for unexpected changes. My principle here was to make it as large as possible, subject to accommodating both native word sizes, in pickling, compiling (including cross compiling), etc. Plus, if it changes, then pickles need to be adapted, so I hope to get it to something we can keep. Do subtract a number that is one less than a multiple of 4, so MaxBytes comes out a multiple of 4. This is pretty confusing. What is really wanted is MaxBytes = (2^31) DIV BITSIZE (Byte) - , but I'm not sure compile-time DIV for this overflow case is well-defined, so 16_7FFFFFFF DIV BITSIZE(Byte) + 1 gives the max multiple of 4 bytes whose bit size is <= LAST(Int32). Then subtract another multiple of 4. So maybe 16_7FFFFFFF DIV BITSIZE(Byte) + 1 - 64 is easier to understand. > Can we declare there is no bound? > No, we're writing code that always ignores the type's bound anyway and uses the cnt field instead. We just want the type's bound as high as possible, to minimize the coding trouble of avoiding the compiler's checks. This is just the kind of low-level coding that the language's safe subset is designed to disallow. > Thank you, > - Jay > > > > Date: Wed, 3 Jun 2015 14:23:07 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] TextLiteral.MaxBytes again (unable to compile 64bit target on 32bit host) > > > > > > > > On 06/02/2015 09:51 PM, Jay K wrote: > > > Ideally: > > > 1 32bit host could target 64bit target > > > 2 32bit texts could be as large as the address space/OS allows. > > > 3 ditto 64bit texts -- larger than 4GB > > > 4 be unpicklable between 32bit and 64bit, as long as they fit into the address space. > > > Currently only #4 is true. > > > TEXTs for any target can only be about 500MB. > > > > > > > > > To fix 2 and 3 requires m3front to use TInt more. > > > > > > This fixes #1 and likely breaks #4. Cutting the limit just a little. > > > Can we special case somehow? > > > Is there some construct we can use that has no stated limit, like 0..cnt - 1? > > > Should/can we introduce one? > > > > > > jbook2:src jay$ git diff "../src/text/TextLiteral.i3" > > > diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 > > > index fa72589..4d16a44 100644 > > > --- a/m3-libs/m3core/src/text/TextLiteral.i3 > > > +++ b/m3-libs/m3core/src/text/TextLiteral.i3 > > > @@ -14,7 +14,7 @@ IMPORT RTHooks, TextClass; > > > CONST > > > (* DIV BITSIZE should not be here! *) > > > (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) > > > - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; > > > + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; > > > > I'd say go ahead and change this. We really do want it to cross-compile > > sooner than anybody will be able to put TInt in all over the place. > > > > I did not build a cross compiler, but did what I think should be a realistic > > simulation of it by changing the cnt:INTEGER field to two INTEGER fields and > > compiling with a 32-bit compiler. That would be the same size as a 64-bit > > INTEGER. It requires MaxBytes <= 16_7FFFFFFF DIV BITSIZE (Byte) - 11 to > > compile, with bit size having space for 31 more bits before it overflows. > > This byte count of buf is a multiple of 4, which we want for unicode-range > > WIDECHAR. Making it -10 overflows, presumably because the additional byte is > > padded out to a multiple of 32 bits. > > > > I would have expected that somewhere, a size including the one-word heap header > > would be computed, but these are never actually heap allocated, and it seems to > > work. But to hedge against things, I suggest using -19. This would allow space > > to count it, cross-compiling 32->64. > > > > This will allow a text literal of 268435436 ISO characters, and a wide text literal > > of 1/4 that, which is not likely to be a serious limitation, considering that it > > all has to be on a single line of source code. > > > > As for pickles, the horse is already out of the barn on that, as pickles could > > already have been written with the current fingerprint, and we can't change the > > type in any way without the fingerprint changing. It is not hard to add > > recognition of the old fingerprint to pickles. Right off hand, I don't remember > > the process for finding actual fingerprint values, but I've done it before and > > can rediscover it. > > > > > (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) > > > (* Do not adjust this for INTEGER size. It makes T have different > > > fingerprints on 32- and 64-bit machines, which undermines pickling/ > > > > > > > > > > > > otherwise we have: > > > > > > > > > new source -> compiling TextLiteral.i3 > > > "../src/text/TextLiteral.i3", line 27: CM3 restriction: record or object type is too large > > > 1 error encountered > > > > > > > > > Ok? > > > - Jay > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Jun 4 16:37:06 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 09:37:06 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> Message-ID: <55706292.9060608@lcwb.coop> On 06/03/2015 06:14 PM, Jay K wrote: > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > An array of chars? Probably has same problem. > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > - Jay > I'm OK with doing it, but I thought you had been talking about using TInt. Would that be better? It would be more general. But LONGINT would be faster at compile time, and less work, since the arithmetic operators would not need to be changed to TInt function calls. There might be quite a lot of those I guess the compiler would be of about equal help in finding missed places to be changed, either way. I just checked, and LONGINT is in the release compiler, contrary to my sense of relative history, so there would be no bootstrap barrier. Either would allow a 32-bit hosted compiler (cross- or native-) to handle types whose byte-count approached 2^31, instead of just 2^23. With a 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, whereas LONGINT would only go to 2^55. Do we really care? > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > From: hosking at purdue.edu > Date: Wed, 3 Jun 2015 14:07:02 -0400 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 4 17:08:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:08:38 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55706292.9060608@lcwb.coop> References: , , , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, , <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , , , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, , <55706292.9060608@lcwb.coop> Message-ID: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:41:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:41:47 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55705A41.7070503@elstel.org> References: , <55705A41.7070503@elstel.org> Message-ID: Do you want 64bit targets to be "better" or "the same"?What if I say the Modula-3 equivalent of, three programs: 1 printf("%d", (int)sizeof(void*)); 2 printf("%d", (int)sizeof(size_t)); 3 printf("%d", (int)sizeof(int)); One could argue in your favor here.The user of void* and size_t is expecting "better".The user of int is expecting "same". Modula-3 could have two integer types, builtin, short names.Or, today, it offers all 4 sizes, you just have to go slightly out of your way. You can say: TYPE int = Ctypes.int; or maybe BasicCtypes.int. and go about your business using int instead of INTEGER andget the compability you desire. Perhaps this should be: int = IntegerTypes.INT32? to not be be C-related? - Jay Date: Thu, 4 Jun 2015 16:01:37 +0200 From: estellnb at elstel.org To: jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem Am 03.06.15 um 03:58 schrieb Jay K: We cannot cross from 32bit host to 64bit target. Ok to commit this? Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. To me this is just another argument for a language specified value range of INTEGER. It makes certain things easier. My suggestion was: TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) BITS 64 FOR INTEGER would only work on x64 systems. The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same tends to be much slower). Regards, Elmar Can't do this. Non-open arrays must have static constant bounds, which the above does not. Open arrays have extra dope and are accessed through two extra pointers, which we really wouldn't want for text literals. TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse that anything else in UNSAFE code. The interface is UNSAFE, which means safe code can't IMPORT it, and thus can't see the declaration. Not very likely that anybody other than the runtime, compiler-generated code, and debugger (written in C anyway) would want to mess with it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:45:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 15:45:00 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, <55706292.9060608@lcwb.coop>, , <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: How strongly preferred and why? good naturedly: 1) to cause me pain so that might give up? 2) to take more time so I don't muck with other stuff? 3) for compatibility with older releases? 4) for easier extension in future? 5) to cause me pain so I whine more about wanting operator overloading? I started this once and it going to be a pain. LONGINT is probably much easier. Maybe I have more patience now. Extension in the future could be addition of INTEGER128 to the language. :) And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128and convert frontend to use it. Hypothetically. Perhaps before that happenswe'll have operator overloading. :) - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 11:32:48 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu Again, TInt preferred. Sent from my iPhone On Jun 4, 2015, at 11:08 AM, Jay K wrote: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 17:54:51 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 4 Jun 2015 08:54:51 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <3A400392-6754-445E-B64F-7E8F1F47985D@purdue.edu> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <3A400392-6754-445E-B64F-7E8F1F47985D@purdue.edu> Message-ID: <119B30EC-1C76-41FB-BCBF-378E287479EF@gmail.com> > I don?t know what the static limit is on an array literal. I suspect all sizes and offsets are limited to bit counts stored INTEGER. So any field or type can be 512mb-1 max on 32bit hosted compiler and 2^56-1 for 64bit hosted compiler. How about a syntax that omits the end of the range? Type t = array [0..] of char? - Jay On Jun 3, 2015, at 5:03 PM, Antony Hosking wrote: > It would not make sense to store an entire blueray movie as a literal?better to structure it. > I don?t know what the static limit is on an array literal. > >> On Jun 3, 2015, at 7:14 PM, Jay K wrote: >> >> sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. >> An array of chars? Probably has same problem. >> >> >> Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? >> >> - Jay >> >> >> >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Wed, 3 Jun 2015 14:07:02 -0400 >> CC: m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> It?s only TEXT literals that are limited here. As in, what appears in a source program. >> >> On Jun 3, 2015, at 1:49 PM, Jay K wrote: >> >> I don't have any, but should we have such artificial limits? >> I'm always nervous about "640k is plenty", "2gig is plenty", "4gb is plenty". >> Thus -- size_t: "4gb is plenty for a 32bit host, but 64bit is unlimited." >> And even that isn't right -- 4bg is often not lenty for a 32bit host. >> File sizes exceed that long ago. >> But "4gb is plenty for 32bit, 64bit is limited" is generally easy to code and I rest there. >> >> >> Given that a blueray movie is an array of bytes, is a text inappropriate? >> >> >> The actual diff presented just adjusts the limit by 8 bytes. >> Ok? >> Or must preserve pickles and can't do this w/o some other change? >> This isn't a more general problem? Or usually pickles >> are 32/64-compatible, but this is a most unusual case? >> >> >> Thanks, >> - Jay >> >> >> >> >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Wed, 3 Jun 2015 13:24:36 -0400 >> CC: m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> Why would you ever want a TEXT literal that is so large? >> By the way, yes TextLiteral is an unsafe interface because of the need to index up to the MaxBytes bound, but the unsafe code is confined to the implementation of the interface. >> A type can be neither unsafe nor safe. It is code that is unsafe or safe. Any code that can see the revealed definition of TextLiteral.T must be unsafe since the interface is unsafe. >> >> On Jun 3, 2015, at 12:36 PM, Jay wrote: >> >> I do this. I shouldn't have to make local edits each time. It isn't meant to only be once. I should be able to long-term use a 32bit toolset to target 32bit & 64bit. >> >> - Jay >> >> On Jun 3, 2015, at 5:16 AM, Antony Hosking wrote: >> >> Other than cross-compile from 32 to 64 bit what purpose does this serve? >> >> Sent from my iPad >> >> On Jun 2, 2015, at 9:58 PM, Jay K wrote: >> >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> diff --git a/m3-libs/m3core/src/text/TextLiteral.i3 b/m3-libs/m3core/src/text/TextLiteral.i3 >> index fa72589..37bf238 100644 >> --- a/m3-libs/m3core/src/text/TextLiteral.i3 >> +++ b/m3-libs/m3core/src/text/TextLiteral.i3 >> @@ -12,9 +12,16 @@ UNSAFE INTERFACE TextLiteral; >> IMPORT RTHooks, TextClass; >> >> CONST >> - (* DIV BITSIZE should not be here! *) >> +(* When cm3 uses TInt and when pickle special cases this, it should be approx: >> + MaxBytes = LAST (INTEGER) - BITSIZE (INTEGER) * 2 *) >> +(* This fails for 32bit host and 64bit target: >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; *) >> +(* Until/unless unpickling special cases this, it must not be sizeof(INTEGER) >> + dependent. *) >> + (* DIV BITSIZE should not be here -- compiler limitation due to >> + non-use of TInt. *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> - MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7; >> + MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 15; >> (* - 8 * ORD(BITSIZE(INTEGER) = 64) *) >> (* Do not adjust this for INTEGER size. It makes T have different >> fingerprints on 32- and 64-bit machines, which undermines pickling/ >> >> >> Yes I know TEXT pickles will be broken. >> I propose that unpickle should special case a few values here. >> Or, isn't the brand supposed to help? >> Or, can we encode this in a better way, w/o an upper bound? >> I know if you put in 0..0 or 0..-1 you incur range violations at runtime. >> Which makes me wonder -- are TEXTs unsafe? >> >> Should we support a syntax something like: >> >> cnt : INTEGER; >> buf : ARRAY [0..cnt - 1] OF Byte; >> ? >> >> Thanks, >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 4 18:35:07 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 11:35:07 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55705A41.7070503@elstel.org> References: <55705A41.7070503@elstel.org> Message-ID: <55707E3B.7020601@lcwb.coop> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: > Am 03.06.15 um 03:58 schrieb Jay K: >> We cannot cross from 32bit host to 64bit target. >> >> >> Ok to commit this? >> >> > > Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. > To me this is just another argument for a language specified value range of INTEGER. > It makes certain things easier. > > My suggestion was: > TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) > while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) > BITS 64 FOR INTEGER would only work on x64 systems. > What you want is already pretty much here. Just declare your desired type yourself: TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; It will always have this range and occupy 32 bits, regardless of the native word size. Pickles will always save and reread it with exactly this range, even between machines of different word size. The only thing that will differ from 32- to 64-bit machine is the size of intermediate results in expressions. If you are already _not_ being careful about avoiding overflows, there are probably cases where the results of silent overflows would differ with machine word size. Not sure what you would really want here. The unsigned arithmetic operations in Word are explicitly defined to silently do binary twos-complement wrap-around when overflows happen. The language does not define this for the builtin signed operators. In code where I care about overflows, I never assume this. It is probably machine-dependent. Or, you can import it: Int32.T. is already declared, in m3core, with this range. However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is almost never what you really want. The only place the BITS specification makes any difference is when you declare a record or object field of this type. In all other situations, BITS 32 FOR .. has _no effect_. The only purpose of BITS is so you can force the layout of a record. In that case, since the range needs exactly 32 bits anyway, even if you omit the BITS, a field of this type will still always occupy 32 bits. But, with the BITS specification, the compiler is not allowed to insert alignment padding, (if it were, you could not fully specify the layout) and if the field comes at a place that is not 32-bit aligned, you will either get a compile-time error or maybe a compiler will accept it and handle unaligned field access (it is not required to--ours does not), neither of which is probably what you want. Without BITS, the compiler will pad as needed. With BITS, you will have to insert explicit padding for alignment yourself as needed, which can be different for every field of this type of every record/object. And later adding/removing/changing any prior field can upset it and require you to rework the padding. Moreover, I am sure there are simple cases where the required padding differs between 32-bit and 64-bit machines. Having been bitten by this more than once, I now only put BITS..FOR as the anonymous type of a specific field, never in a named type to be used in multiple places. (Actually, BITS does the same thing when used as the type of an array element, but cases where this matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array fits in a single machine word. I do have a couple of such cases.) > The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value > range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for > any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). > > Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in > here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In > a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same > tends to be much slower). > > Regards, > Elmar > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Thu Jun 4 19:43:07 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Thu, 04 Jun 2015 19:43:07 +0200 Subject: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? In-Reply-To: <55707E3B.7020601@lcwb.coop> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> Message-ID: <55708E2B.4040607@elstel.org> Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem Am 04.06.15 um 18:35 schrieb Rodney M. Bates: > > > On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: >> Am 03.06.15 um 03:58 schrieb Jay K: >>> We cannot cross from 32bit host to 64bit target. >>> >>> >>> Ok to commit this? >>> >>> >> >> Compatibility between the 32bit and the 64bit version of the same >> program would be the minimum I expected from a pickle. >> To me this is just another argument for a language specified value >> range of INTEGER. >> It makes certain things easier. >> >> My suggestion was: >> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS >> 32 FOR INTEGER for 32 and 64bit targets) >> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I >> believe there is no reason to forbid it for INTEGER) >> BITS 64 FOR INTEGER would only work on x64 systems. >> > > What you want is already pretty much here. Just declare your desired > type yourself: > > TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; This should be good news. However my suggestion was to make Int32.T the default as this will be somewhat faster for the average application which is data intensive (just think of image processing) which will never be rewritten to use Int32.T instead of INT. All of our current and old programs being prepared to run on a 32bit as well as a 64bit target should compile and run without any problem. There is one single exception where the compiler would throw an error: when using INTEGER for address arithmetics in unsafe modules (error is: can only LOOPHOLE between types of the same size, or some way similar as I have already tested this). There we would need the so called targetint/offset. If I understand things right the following two things would need to get implemented to allow a default int size of 32 for existing applications (be it the default or just an additional compiler option as the size of WIDECHAR already is): * automatic range adaption for packed ints (i.e. those with BITS .. FOR) * allowing non 32bit alignment of INTs in memory (as is already the case for WIDE[CHAR]s) > > It will always have this range and occupy 32 bits, regardless of the > native word size. Pickles > will always save and reread it with exactly this range, even between > machines of different > word size. So the target size of the pickle is already determined by the respective INT subrange, right? Something that should to my believe be well documented as one could easily like to extend an enum with 255 members to more or equal than 256 an then wonder why the new an the old file formats become incompatible. To circumvent such a problem we would likely need to allow BITS 8*anything FOR INTEGER and take the bitcount specified in BITS for in order to determine the target size of an object instead of semi-automatically readjusting it all the time. > > The only thing that will differ from 32- to 64-bit machine is the size > of intermediate results in > expressions. If you are already _not_ being careful about avoiding > overflows, there are probably > cases where the results of silent overflows would differ with machine > word size. Not sure what > you would really want here. The unsigned arithmetic operations in > Word are explicitly defined > to silently do binary twos-complement wrap-around when overflows > happen. The language does not > define this for the builtin signed operators. In code where I care > about overflows, I never assume > this. It is probably machine-dependent. Yes, I believe it would be good like this. No real 32bit INT with 32bit arithmetics but just restricting the range for Int32.T when it gets written into memory. > > Or, you can import it: Int32.T. is already declared, in m3core, with > this range. > > However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is > almost never what you really want. > The only place the BITS specification makes any difference is when you > declare a record or object > field of this type. In all other situations, BITS 32 FOR .. has _no > effect_. The only purpose of > BITS is so you can force the layout of a record. In that case, since > the range needs exactly 32 > bits anyway, even if you omit the BITS, a field of this type will > still always occupy 32 bits. Nonetheless it is currently not possible to declare a variable of a 32bit type globally: TYPE Pixel = BITS 32 FOR INTEGER; unluckily does not work as we can have no global variable of type Pixel. This is not only un-orthogonal but also a very practical problem; Suggesting that we call procedures with a parameter of type pixel we will never be able to change that type f.i. to a packed record with members red, green, blue alpha if there are places in our main program where we need to declare VAR pix1, pix2 : INTEGER just because Pixel is a packed type being refused as global variable. > > But, with the BITS specification, the compiler is not allowed to > insert alignment padding, (if it > were, you could not fully specify the layout) and if the field comes > at a place that is not 32-bit > aligned, you will either get a compile-time error or maybe a compiler > will accept it and handle > unaligned field access (it is not required to--ours does not), neither > of which is probably what > you want. Without BITS, the compiler will pad as needed. With BITS, > you will have to insert explicit > padding for alignment yourself as needed, which can be different for > every field of this type of every > record/object. And later adding/removing/changing any prior field can > upset it and require you to > rework the padding. Moreover, I am sure there are simple cases where > the required padding differs > between 32-bit and 64-bit machines. On the long term I would personally even advocate an ALIGN directive: f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END; It would be very handy to have this for tapping the full power of the vectorization units of current processors. Manually re-aligning heap allocated objects in an unsafe module is not a very satisfying alternative (no local/global/stack allocation, memory waste and thus a certain performance loss if there are many wholes). However this would possibly also make changes to our garbage collector necessary as it currently only guarantees word alignment. Finally it would also be possible to write ALIGN BITSIZE(TARGETINT) .... > > Having been bitten by this more than once, I now only put BITS..FOR as > the anonymous type of a specific > field, never in a named type to be used in multiple places. A less bitchy and more orthogonal Modula-3 compiler supporting integer globals and locals of non word size and alignment would be a real pleasure, do you consent Rodney? > > (Actually, BITS does the same thing when used as the type of an array > element, but cases where this > matters are rare. If the bit count evenly divides the word size, > there won't be any alignment padding > needed, and BITS won't matter. Otherwise, the compiler is likely to > refuse, unless the entire array > fits in a single machine word. I do have a couple of such cases.) > >> The value range - bitsize correlation problem could be solved easily >> by automatically extending and reducing the value >> range of an integer type to what its binary representation allows as >> long as no explicit range has ever been specified for >> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR >> [-32768..+32767]). >> >> Finally it needs to be said that the argument that target size >> arithmetics is always the fastest which was often used in >> here can be very wrong. It does not hold for the 64bit systems of >> today because the CPU is no more the bottleneck. In >> a fact now the memory hierarchy has become the new bottleneck (which >> means that using more memory for the same >> tends to be much slower). >> >> Regards, >> Elmar >> > From mika at async.caltech.edu Thu Jun 4 19:46:29 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 04 Jun 2015 10:46:29 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55707E3B.7020601@lcwb.coop> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> Message-ID: <20150604174629.603F21A2065@async.async.caltech.edu> >(Actually, BITS does the same thing when used as the type of an array element, but cases where this >matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >fits in a single machine word. I do have a couple of such cases.) For a lot of hardware-related problems, the following type is quite useful: ARRAY [..] OF BITS 1 FOR BOOLEAN (and works with our compilers, too, as far as I know.) Mika From rodney_bates at lcwb.coop Thu Jun 4 19:51:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 12:51:43 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: <5570902F.3030401@lcwb.coop> On 06/04/2015 11:50 AM, Antony Hosking wrote: > >> On Jun 4, 2015, at 11:45 AM, Jay K > wrote: >> >> How strongly preferred and why? > > TInt preferred because LONGINT is still a terrible hack, and I hate to see it pollute the system more than it already has. > What?s more, the backends break on a number of operations for it. Danger, Will Robinson! This could introduce truly show-stopping bootstrap problems, using LONGINT in the compiler, and in places essential to its ability to function at all. One little bug and we could end up with neither a chicken nor an egg. > I now regret ever having introduced it, and would like to back it out. We would have been better to specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF INTEGER, and then hidden manipulation of those offsets behind a clean interface, much as Tint does. I think the one use-case we had for it to allow interfacing directly to C functions that take offset_t arguments could have been finessed differently. > >> good naturedly: >> >> >> 1) to cause me pain so that might give up? >> 2) to take more time so I don't muck with other stuff? >> 3) for compatibility with older releases? >> 4) for easier extension in future? >> 5) to cause me pain so I whine more about wanting operator overloading? > > Perhaps 3 & 4. > >> I started this once and it going to be a pain. >> LONGINT is probably much easier. >> Maybe I have more patience now. > > :-) Patience accrues with age? > >> Extension in the future could be addition of INTEGER128 to the language. :) >> And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128 >> and convert frontend to use it. Hypothetically. Perhaps before that happens >> we'll have operator overloading. :) > > Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. > >> >> >> - Jay >> >> >> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- --- >> Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> From: hosking at purdue.edu >> Date: Thu, 4 Jun 2015 11:32:48 -0400 >> CC: rodney.m.bates at acm.org ; m3devel at elegosoft.com >> To: jay.krell at cornell.edu >> >> Again, TInt preferred. >> >> Sent from my iPhone >> >> On Jun 4, 2015, at 11:08 AM, Jay K > wrote: >> >> When I proposed TInt I had actually forgotten about LONGINT! >> LONGINT is easier to code to (until/unless we get operator overloading...) >> >> TInt is easier to extend, and is portable to before the current release. >> I think LONGINT is adequate. >> I did extent TInt to 72 bits I think. >> >> - Jay >> >> >> > Date: Thu, 4 Jun 2015 09:37:06 -0500 >> > From:rodney_bates at lcwb.coop >> > To:jay.krell at cornell.edu ;hosking at purdue.edu >> > CC:m3devel at elegosoft.com >> > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> > >> > >> > >> > On 06/03/2015 06:14 PM, Jay K wrote: >> > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. >> > > An array of chars? Probably has same problem. >> > > >> > > >> > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? >> > > >> > > - Jay >> > > >> > >> > I'm OK with doing it, but I thought you had been talking about using TInt. >> > Would that be better? It would be more general. But LONGINT would be >> > faster at compile time, and less work, since the arithmetic operators >> > would not need to be changed to TInt function calls. There might >> > be quite a lot of those I guess the compiler would be of about equal >> > help in finding missed places to be changed, either way. >> > >> > I just checked, and LONGINT is in the release compiler, contrary to >> > my sense of relative history, so there would be no bootstrap barrier. >> > >> > Either would allow a 32-bit hosted compiler (cross- or native-) to handle >> > types whose byte-count approached 2^31, instead of just 2^23. With a >> > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, >> > whereas LONGINT would only go to 2^55. Do we really care? >> > >> > > >> > > >> > > >> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----- >> > -- >> > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem >> > > From:hosking at purdue.edu >> > > Date: Wed, 3 Jun 2015 14:07:02 -0400 >> > > CC:m3devel at elegosoft.com >> > > To:jay.krell at cornell.edu >> > > >> > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. >> > > >> > > >> > -- >> > Rodney Bates >> >rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From adacore at marino.st Thu Jun 4 19:57:23 2015 From: adacore at marino.st (John Marino) Date: Thu, 04 Jun 2015 19:57:23 +0200 Subject: [M3devel] rpath woes Message-ID: <55709183.4060804@marino.st> I'm been finished with my port update for more than 1 day, except for one tiny problem. The rpath was wrong after I shifted the installation to /usr/local/cm3. I have tried just about every combination of changes to files at m3-sys/cminstall/src/config-no-install and I can't get the rpath to change. It's set at "/usr/local/lib", I don't know where it comes from. I want it to be "/usr/local/cm3/lib:/usr/local/lib". At gnuld.common, there are weird definitions of "-Wl,-rpath," but altering them doesn't help (not even "-W1,-rpath,//$ORIGIN"/../lib" So I am at a loss where rpath is getting set. If I knew that, I could change it. It seems to ignore all this SYSTEM_LD stuff so it must not be set. Any hints or do I have to keep randomly changing anything like looks like it could be a linker flag? John From jay.krell at cornell.edu Thu Jun 4 20:01:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 18:01:05 +0000 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: , , <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com>, <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu>, , <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu>, <55706292.9060608@lcwb.coop>, , <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu>, , Message-ID: > Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. The combination of features I want hasn't been materialized. Something like: 1) operator overloading -- C++ 2) templates with type deduction (i.e. C++ function templates, not just class templates or generic modules) -- C++ 3) a small easy to understand language -- not C++, maybe Modula-3 4) a small easy to understand compiler (m3front I don't understand) -- nowhere 5) fast compilation -- Modula-3 6) optional safey -- Modula-3 7) multiple inheritance, at least of "interfaces" -- C++ 8) optionally stack or inline allocation of "classes"/"objects" -- C++ 9) clear choice of build system -- Modula-3 perhaps 10) locals with destructors -- C++ 11) clear portable choice for remoting/RPC -- Modula-3 perhaps 12) needs backend work -- Modula-3 perhaps 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta 14) ?safety w/o garbage collection? -- rust?? 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. Not Python/Perl/Ruby/Erlang/sh/Tcl. 16) static compilation to native code, maybe -- C++ and Modula-3! Go? Net.Native? 17) non-virtual member functions -- not C or Modula-3 18) virtual member functions -- C++, Modula-3 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ was late here. 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 and C are decades late. 22) good string library 23) good regex library Basically, for now, I want static compilation to native code, fast compilation, optional safety. That combination is rare. Rust is unusual in safety w/o gc. Extended static lifetime analysis/verification... - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 12:50:02 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu On Jun 4, 2015, at 11:45 AM, Jay K wrote: How strongly preferred and why? TInt preferred because LONGINT is still a terrible hack, and I hate to see it pollute the system more than it already has. What?s more, the backends break on a number of operations for it. I now regret ever having introduced it, and would like to back it out. We would have been better to specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF INTEGER, and then hidden manipulation of those offsets behind a clean interface, much as Tint does. I think the one use-case we had for it to allow interfacing directly to C functions that take offset_t arguments could have been finessed differently. good naturedly: 1) to cause me pain so that might give up? 2) to take more time so I don't muck with other stuff? 3) for compatibility with older releases? 4) for easier extension in future? 5) to cause me pain so I whine more about wanting operator overloading? Perhaps 3 & 4. I started this once and it going to be a pain. LONGINT is probably much easier. Maybe I have more patience now. :-) Patience accrues with age? Extension in the future could be addition of INTEGER128 to the language. :)And the 128 bit targets will initially have 64bit LONGINT limits, until we add INTEGER128and convert frontend to use it. Hypothetically. Perhaps before that happenswe'll have operator overloading. :) Operator overloading is anathema to the Modula-3 design principles. Use C++ if that is what you want. - Jay Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem From: hosking at purdue.edu Date: Thu, 4 Jun 2015 11:32:48 -0400 CC: rodney.m.bates at acm.org; m3devel at elegosoft.com To: jay.krell at cornell.edu Again, TInt preferred. Sent from my iPhone On Jun 4, 2015, at 11:08 AM, Jay K wrote: When I proposed TInt I had actually forgotten about LONGINT!LONGINT is easier to code to (until/unless we get operator overloading...) TInt is easier to extend, and is portable to before the current release.I think LONGINT is adequate.I did extent TInt to 72 bits I think. - Jay > Date: Thu, 4 Jun 2015 09:37:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > sorry, I missed that it is literals...so I can convert a blueray movie into a source file containing the data all in a text? Far fetched. > > An array of chars? Probably has same problem. > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and field offsets? > > > > - Jay > > > > I'm OK with doing it, but I thought you had been talking about using TInt. > Would that be better? It would be more general. But LONGINT would be > faster at compile time, and less work, since the arithmetic operators > would not need to be changed to TInt function calls. There might > be quite a lot of those I guess the compiler would be of about equal > help in finding missed places to be changed, either way. > > I just checked, and LONGINT is in the release compiler, contrary to > my sense of relative history, so there would be no bootstrap barrier. > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > types whose byte-count approached 2^31, instead of just 2^23. With a > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > From: hosking at purdue.edu > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > CC: m3devel at elegosoft.com > > To: jay.krell at cornell.edu > > > > It?s only TEXT /literals/ that are limited here. As in, what appears in a source program. > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 4 20:05:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 4 Jun 2015 18:05:00 +0000 Subject: [M3devel] rpath woes In-Reply-To: <55709183.4060804@marino.st> References: <55709183.4060804@marino.st> Message-ID: My understanding: The modula-3 libraries, e.g. m3core,are found from the rpath I put in, like $ORIGIN:$ORIGIN/../lib Other libraries like libc.so are found by full path, that is already correct at build time? Do users want to possibly have a private /usr/local/lib/libc.so?I hadn't considered that. Or build is "staged" and the full paths aren't correct at build time?I think I hadn't considered that either. You are on the right track, looking in this area. Did you notice the "portable rpath" thingy that make-dist.py changes? - Jay > Date: Thu, 4 Jun 2015 19:57:23 +0200 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] rpath woes > > I'm been finished with my port update for more than 1 day, except for > one tiny problem. The rpath was wrong after I shifted the installation > to /usr/local/cm3. > > I have tried just about every combination of changes to files at > m3-sys/cminstall/src/config-no-install and I can't get the rpath to change. > > It's set at "/usr/local/lib", I don't know where it comes from. I want > it to be "/usr/local/cm3/lib:/usr/local/lib". > > At gnuld.common, there are weird definitions of "-Wl,-rpath," but > altering them doesn't help (not even "-W1,-rpath,//$ORIGIN"/../lib" > > So I am at a loss where rpath is getting set. If I knew that, I could > change it. It seems to ignore all this SYSTEM_LD stuff so it must not > be set. > > Any hints or do I have to keep randomly changing anything like looks > like it could be a linker flag? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Thu Jun 4 20:10:07 2015 From: adacore at marino.st (John Marino) Date: Thu, 04 Jun 2015 20:10:07 +0200 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st> Message-ID: <5570947F.2080600@marino.st> On 6/4/2015 20:05, Jay K wrote: > My understanding: > The modula-3 libraries, e.g. m3core, > are found from the rpath I put in, like > $ORIGIN:$ORIGIN/../lib > Other libraries like libc.so are found by full path, that is already > correct at build time? > Do users want to possibly have a private /usr/local/lib/libc.so? > I hadn't considered that. > Or build is "staged" and the full paths aren't correct at build time? > I think I hadn't considered that either. > You are on the right track, looking in this area. > Did you notice the "portable rpath" thingy that make-dist.py changes? I believe make-dist.py is hard-coding portable rpath which means it's hardcoding $ORIGIN I think, but it's not working. The origin flag is set, but $ORIGIN itself is not. I think this is a bug. We prefer to use hardcoded rpaths in ports, but I will settle for $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not NetBSD (not sure about OpenBSD, I'd guess it does though). The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't work either. John From hendrik at topoi.pooq.com Thu Jun 4 20:24:44 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 4 Jun 2015 14:24:44 -0400 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st> Message-ID: <20150604182444.GB8703@topoi.pooq.com> On Thu, Jun 04, 2015 at 06:05:00PM +0000, Jay K wrote: > Do users want to possibly have a private /usr/local/lib/libc.so?I > hadn't considered that. The musl users might. It's another C runtime system which may have some standards-compliance and efficiency advantages. see http://www.musl-libc.org/ -- hemdrik From lists at darko.org Thu Jun 4 20:32:17 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 4 Jun 2015 11:32:17 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: You really should try writing your own language, it's really not that hard. M3 will never give you satisfaction. You've already implemented a C back end and no doubt have an idea of how a compiler works. As for language features, I think we should be looking for features to remove. On Thu, Jun 4, 2015 at 11:01 AM, Jay K wrote: > > Operator overloading is anathema to the Modula-3 design principles. > Use C++ if that is what you want. > > > The combination of features I want hasn't been materialized. > Something like: > > 1) operator overloading -- C++ > 2) templates with type deduction (i.e. C++ function templates, not just > class templates or generic modules) -- C++ > 3) a small easy to understand language -- not C++, maybe Modula-3 > 4) a small easy to understand compiler (m3front I don't understand) -- > nowhere > 5) fast compilation -- Modula-3 > 6) optional safey -- Modula-3 > 7) multiple inheritance, at least of "interfaces" -- C++ > 8) optionally stack or inline allocation of "classes"/"objects" -- C++ > 9) clear choice of build system -- Modula-3 perhaps > 10) locals with destructors -- C++ > 11) clear portable choice for remoting/RPC -- Modula-3 perhaps > 12) needs backend work -- Modula-3 perhaps > 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta > 14) ?safety w/o garbage collection? -- rust?? > 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. > Not Python/Perl/Ruby/Erlang/sh/Tcl. > 16) static compilation to native code, maybe -- C++ and Modula-3! Go? > Net.Native? > 17) non-virtual member functions -- not C or Modula-3 > 18) virtual member functions -- C++, Modula-3 > 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) > 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ > was late here. > 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 > and C are decades late. > 22) good string library > 23) good regex library > > > Basically, for now, I want static compilation to native code, fast > compilation, optional safety. > That combination is rare. > > > Rust is unusual in safety w/o gc. Extended static lifetime > analysis/verification... > > > - Jay > > > > ------------------------------ > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > From: hosking at purdue.edu > Date: Thu, 4 Jun 2015 12:50:02 -0400 > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > To: jay.krell at cornell.edu > > > On Jun 4, 2015, at 11:45 AM, Jay K wrote: > > How strongly preferred and why? > > > TInt preferred because LONGINT is still a terrible hack, and I hate to see > it pollute the system more than it already has. What?s more, the backends > break on a number of operations for it. I now regret ever having > introduced it, and would like to back it out. We would have been better to > specialize 32-bit targets to treat 64-bit offsets as ARRAY [0..1] OF > INTEGER, and then hidden manipulation of those offsets behind a clean > interface, much as Tint does. I think the one use-case we had for it to > allow interfacing directly to C functions that take offset_t arguments > could have been finessed differently. > > good naturedly: > > > 1) to cause me pain so that might give up? > 2) to take more time so I don't muck with other stuff? > 3) for compatibility with older releases? > 4) for easier extension in future? > 5) to cause me pain so I whine more about wanting operator overloading? > > > Perhaps 3 & 4. > > I started this once and it going to be a pain. > LONGINT is probably much easier. > Maybe I have more patience now. > > > :-) Patience accrues with age? > > Extension in the future could be addition of INTEGER128 to the language. :) > And the 128 bit targets will initially have 64bit LONGINT limits, until we > add INTEGER128 > and convert frontend to use it. Hypothetically. Perhaps before that happens > we'll have operator overloading. :) > > > Operator overloading is anathema to the Modula-3 design principles. Use > C++ if that is what you want. > > > > - Jay > > > ------------------------------ > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > From: hosking at purdue.edu > Date: Thu, 4 Jun 2015 11:32:48 -0400 > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > To: jay.krell at cornell.edu > > Again, TInt preferred. > > Sent from my iPhone > > On Jun 4, 2015, at 11:08 AM, Jay K wrote: > > When I proposed TInt I had actually forgotten about LONGINT! > LONGINT is easier to code to (until/unless we get operator overloading...) > > TInt is easier to extend, and is portable to before the current release. > I think LONGINT is adequate. > I did extent TInt to 72 bits I think. > > - Jay > > > > Date: Thu, 4 Jun 2015 09:37:06 -0500 > > From: rodney_bates at lcwb.coop > > To: jay.krell at cornell.edu; hosking at purdue.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > > > > > > > > On 06/03/2015 06:14 PM, Jay K wrote: > > > sorry, I missed that it is literals...so I can convert a blueray movie > into a source file containing the data all in a text? Far fetched. > > > An array of chars? Probably has same problem. > > > > > > > > > Can I/we please LONGINT-ize the compiler now, for all type sizes and > field offsets? > > > > > > - Jay > > > > > > > I'm OK with doing it, but I thought you had been talking about using > TInt. > > Would that be better? It would be more general. But LONGINT would be > > faster at compile time, and less work, since the arithmetic operators > > would not need to be changed to TInt function calls. There might > > be quite a lot of those I guess the compiler would be of about equal > > help in finding missed places to be changed, either way. > > > > I just checked, and LONGINT is in the release compiler, contrary to > > my sense of relative history, so there would be no bootstrap barrier. > > > > Either would allow a 32-bit hosted compiler (cross- or native-) to handle > > types whose byte-count approached 2^31, instead of just 2^23. With a > > 64-bit hosted compiler, TInt would handle 2^63 byte-sized objects, > > whereas LONGINT would only go to 2^55. Do we really care? > > > > > > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > -- > > > Subject: Re: [M3devel] 32bit host 64bit target TextLiteral recurring > problem > > > From: hosking at purdue.edu > > > Date: Wed, 3 Jun 2015 14:07:02 -0400 > > > CC: m3devel at elegosoft.com > > > To: jay.krell at cornell.edu > > > > > > It?s only TEXT /literals/ that are limited here. As in, what appears > in a source program. > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 5 00:24:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 04 Jun 2015 17:24:26 -0500 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <20150604174629.603F21A2065@async.async.caltech.edu> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> <20150604174629.603F21A2065@async.async.caltech.edu> Message-ID: <5570D01A.5000909@lcwb.coop> On 06/04/2015 12:46 PM, mika at async.caltech.edu wrote: >> (Actually, BITS does the same thing when used as the type of an array element, but cases where this >> matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >> needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >> fits in a single machine word. I do have a couple of such cases.) > > For a lot of hardware-related problems, the following type is quite useful: > > ARRAY [..] OF BITS 1 FOR BOOLEAN > > (and works with our compilers, too, as far as I know.) > Yeah, I forgot about that group of cases. Without a BITS spec, the only sizes the compiler will choose are 64, 32, 16, or 8. If you want 4, 2, or 1, or something else that doesn't divide 32, you have to use BITS. But they will work fine. > Mika > -- Rodney Bates rodney.m.bates at acm.org From adacore at marino.st Fri Jun 5 00:28:36 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 00:28:36 +0200 Subject: [M3devel] rpath woes In-Reply-To: <5570947F.2080600@marino.st> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> Message-ID: <5570D114.40104@marino.st> On 6/4/2015 20:10, John Marino wrote: > On 6/4/2015 20:05, Jay K wrote: >> My understanding: >> The modula-3 libraries, e.g. m3core, >> are found from the rpath I put in, like > >> $ORIGIN:$ORIGIN/../lib >> Other libraries like libc.so are found by full path, that is already >> correct at build time? >> Do users want to possibly have a private /usr/local/lib/libc.so? >> I hadn't considered that. >> Or build is "staged" and the full paths aren't correct at build time? >> I think I hadn't considered that either. >> You are on the right track, looking in this area. >> Did you notice the "portable rpath" thingy that make-dist.py changes? > > I believe make-dist.py is hard-coding portable rpath which means it's > hardcoding $ORIGIN I think, but it's not working. > > The origin flag is set, but $ORIGIN itself is not. I think this is a bug. > > We prefer to use hardcoded rpaths in ports, but I will settle for > $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not > NetBSD (not sure about OpenBSD, I'd guess it does though). > > The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't > work either. Okay, I have this solved. The problem is FREEBSD.common is obsolete (origin linker flags were wrong) I basically copied LINUX.common contents to it and now everything works as expected. I've updated the port here: http://www.freshports.org/lang/modula3/ The patch for FREEBSD.common is here: https://svnweb.freebsd.org/ports/head/lang/modula3/files/ I think all references to FREEBSD4 could be removed (as I did in the patch). FreeBSD4 was EOL in Jan 2007. So I'm basically done until its time to change the port to use the C backend (and/or bootstrap it to DragonFly) John From jay.krell at cornell.edu Fri Jun 5 06:16:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 5 Jun 2015 04:16:24 +0000 Subject: [M3devel] rpath woes In-Reply-To: <5570D114.40104@marino.st> References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>,<5570D114.40104@marino.st> Message-ID: The FreeBSD4 users were surprisingly vocal surprisingly recently. So I put some work into it. I agree it is an old system. Linux.common and FreeBSD.common look very similar but I agree different and if it is working for you, good, I can ignore it? I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? Was this right for FreeBSD 5 or 6 or 7 or 8? Dragonfly should be fairly easy...though it is a bit more work than is ideal. The main thing is we still have lists of targets. Look in m3-sys/m3middle/src/Target.* Soon hopefully the jmpbuf there will go away. That will help. Areas to port are easy to find with grep/findstr: C:\dev2\cm3\m3-libs\libm3>findstr /m /s /i OPENBSD * src\os\POSIX\m3makefile src\os\POSIX\m3makefile-old src\uid\POSIX\getMID.c src\uid\POSIX\MachineIDPosixC.c tests\random\src\m3makefile C:\dev2\cm3\m3-libs\m3core>findstr /m /s /i OPENBSD * src\atomic\m3makefile src\C\Common\Cstring.i3 src\Csupport\Common\lround-readme.txt src\Csupport\Common\s_llroundl.c src\Csupport\Common\s_lround.c src\Csupport\Common\s_lroundl.c src\Csupport\m3makefile-old src\float\m3makefile src\float\m3makefile-old src\m3core.h src\runtime\POSIX\RTSignalC.c src\thread\POSIX\ThreadPosixC.c src\thread\PTHREAD\m3makefile src\thread\PTHREAD\ThreadOpenBSD.c src\thread\PTHREAD\ThreadPThreadC.c src\time\POSIX\m3makefile.old src\unix\Common\Ustat.i3 src\unix\m3makefile src\unix\m3makefile-old C:\dev2\cm3\m3-sys\m3middle>findstr /m /s /i OPENBSD * src\Target.i3 src\Target.i3-old src\Target.m3 src\Target.m3-old This looks scarier than it is. Many cases are just to add to a list of platforms you are close enough to. Some cases I hope to eliminate -- cooperative suspend will cut out a swath for example. Thanks, - Jay > Date: Fri, 5 Jun 2015 00:28:36 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > > On 6/4/2015 20:10, John Marino wrote: > > On 6/4/2015 20:05, Jay K wrote: > >> My understanding: > >> The modula-3 libraries, e.g. m3core, > >> are found from the rpath I put in, like > > > >> $ORIGIN:$ORIGIN/../lib > >> Other libraries like libc.so are found by full path, that is already > >> correct at build time? > >> Do users want to possibly have a private /usr/local/lib/libc.so? > >> I hadn't considered that. > >> Or build is "staged" and the full paths aren't correct at build time? > >> I think I hadn't considered that either. > >> You are on the right track, looking in this area. > >> Did you notice the "portable rpath" thingy that make-dist.py changes? > > > > I believe make-dist.py is hard-coding portable rpath which means it's > > hardcoding $ORIGIN I think, but it's not working. > > > > The origin flag is set, but $ORIGIN itself is not. I think this is a bug. > > > > We prefer to use hardcoded rpaths in ports, but I will settle for > > $ORIGIN if it works. $ORIGIN works on FreeBSD and DragonFly, but not > > NetBSD (not sure about OpenBSD, I'd guess it does though). > > > > The "-Wl,rpath," looks wrong to me, but sticking $ORIGIN here didn't > > work either. > > Okay, I have this solved. > The problem is FREEBSD.common is obsolete (origin linker flags were > wrong) I basically copied LINUX.common contents to it and now > everything works as expected. > > I've updated the port here: > http://www.freshports.org/lang/modula3/ > > The patch for FREEBSD.common is here: > https://svnweb.freebsd.org/ports/head/lang/modula3/files/ > > I think all references to FREEBSD4 could be removed (as I did in the > patch). FreeBSD4 was EOL in Jan 2007. > > So I'm basically done until its time to change the port to use the C > backend (and/or bootstrap it to DragonFly) > > John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jun 5 07:50:33 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 07:50:33 +0200 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>, <5570D114.40104@marino.st> Message-ID: <557138A9.3000605@marino.st> On 6/5/2015 06:16, Jay K wrote: > The FreeBSD4 users were surprisingly vocal surprisingly recently. > So I put some work into it. > I agree it is an old system. It's completely unsupported (for 7 years now) and they don't need the latest M3 right? Old release still work fine, that should be enough justification to change this on trunk. > Linux.common and FreeBSD.common look very similar but I agree different > and if it is working for you, good, I can ignore it? > I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? > Was this right for FreeBSD 5 or 6 or 7 or 8? yes. I am using the latest binutils (thus latest ld) with M3. The base ld is ancient and hand maintained (a GPLv3 issue) so I don't know, Actually, the linux way is opposite: It explicitly defines SYSTEM_LD rather than use flags for SYSTEM_CC. So rather than flags, we are talking about the approach. Really nobody with a supported FreeBSD release needs to build from scratch because it exists in ports. If anybody absolutely wants to, just say "emulates what the port is doing". This is actually easier said than done because in addition to using the latest binutils, there are a number of inline substitutions that are done, see post-patch target here, starting line 78: https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup line 85 is where newer gcc and newer as is set for m3. John From adacore at marino.st Fri Jun 5 08:10:08 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 08:10:08 +0200 Subject: [M3devel] install issues (stripping, permissions) Message-ID: <55713D40.6080305@marino.st> This is old news but I never reported it. See the install target on the port's makefile (starting line 100): https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup I have to due numerous changes to the installed files to make it conform to ports standards. The issues involved stripping and permissions. For ports, executables are installed stripped unless explicitly wanted. Usually gnu makefiles support this with "install" and "install-strip" targets. This can be done after the fact, as I have done, with the ${STRIP_CMD} command, which does nothing when stripping is not desired. The other issue is that there were numerous files in pkg/ that were executable and shouldn't have been. I used find/chmod to remove exec perms for all except "cm3" file. I did this 1.5 years ago, I'm now wondering how right it was. The current list: pkg/m3core/src/C/Common/Csetjmp.i3 pkg/m3back/src/M3C.i3 pkg/m3staloneback/AMD64_FREEBSD/m3back pkg/cm3/AMD64_FREEBSD/cm3 pkg/libdump/AMD64_FREEBSD/libdump pkg/cmpfp/AMD64_FREEBSD/cmpfp pkg/formsview/AMD64_FREEBSD/formsview pkg/vorun/AMD64_FREEBSD/vorun pkg/pkl-fonts/AMD64_FREEBSD/PklFonts pkg/hack/AMD64_FREEBSD/dummy pkg/test/AMD64_FREEBSD/test Should there be executables in pkg/ ? Is the chmod command then wrong? I had to strips these as well to pass QA tests. John From wagner at elegosoft.com Fri Jun 5 09:26:18 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 5 Jun 2015 09:26:18 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55713D40.6080305@marino.st> References: <55713D40.6080305@marino.st> Message-ID: <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> On Fri, 05 Jun 2015 08:10:08 +0200 John Marino wrote: > The other issue is that there were numerous files in pkg/ that were > executable and shouldn't have been. I used find/chmod to remove exec > perms for all except "cm3" file. I did this 1.5 years ago, I'm now > wondering how right it was. The current list: > > pkg/m3core/src/C/Common/Csetjmp.i3 > pkg/m3back/src/M3C.i3 > pkg/m3staloneback/AMD64_FREEBSD/m3back > pkg/cm3/AMD64_FREEBSD/cm3 > pkg/libdump/AMD64_FREEBSD/libdump > pkg/cmpfp/AMD64_FREEBSD/cmpfp > pkg/formsview/AMD64_FREEBSD/formsview > pkg/vorun/AMD64_FREEBSD/vorun > pkg/pkl-fonts/AMD64_FREEBSD/PklFonts > pkg/hack/AMD64_FREEBSD/dummy > pkg/test/AMD64_FREEBSD/test > > Should there be executables in pkg/ ? Is the chmod command then wrong? > I had to strips these as well to pass QA tests. Except for the i3-files these are all programs and should be executable. The builder ships programs to the package target directory, and Programs with a capital P to the bin directory. As I said, M3 packages don't fit well into OS packages. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From estellnb at elstel.org Fri Jun 5 10:21:31 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:21:31 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <7427033D-A18B-4F81-A823-229F91313F16@purdue.edu> References: <55705A41.7070503@elstel.org> <7427033D-A18B-4F81-A823-229F91313F16@purdue.edu> Message-ID: <55715C0B.8090207@elstel.org> Antony, you do not seem to understand what I have proposed. Please have a look at my latest mail about Re: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? Am 04.06.15 um 19:14 schrieb Antony Hosking: > >> On Jun 4, 2015, at 10:01 AM, Elmar Stellnberger > > wrote: >> >> Am 03.06.15 um 03:58 schrieb Jay K: >>> We cannot cross from 32bit host to 64bit target. >>> >>> >>> Ok to commit this? >>> >>> >> >> Compatibility between the 32bit and the 64bit version of the same >> program would be the minimum I expected from a pickle. >> To me this is just another argument for a language specified value >> range of INTEGER. >> It makes certain things easier. >> >> My suggestion was: >> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS >> 32 FOR INTEGER for 32 and 64bit targets) >> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I >> believe there is no reason to forbid it for INTEGER) >> BITS 64 FOR INTEGER would only work on x64 systems. > > You are still misunderstanding the purpose of BITS FOR. It is used to > express packing requirements for use of those types *inside* > structured values (such as RECORD or ARRAY) to enforce a particular > bit-packing and to avoid alignment contraints. What it means is that > bit-wise operations must be used to load/store the value in memory in > the case that aligned operations do not suffice. > >> >> The value range - bitsize correlation problem could be solved easily >> by automatically extending and reducing the value >> range of an integer type to what its binary representation allows as >> long as no explicit range has ever been specified for >> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR >> [-32768..+32767]). > > A value of type [-32768..+32767] *never* occupies more than 16 bits in > memory! > >> Finally it needs to be said that the argument that target size >> arithmetics is always the fastest which was often used in >> here can be very wrong. It does not hold for the 64bit systems of >> today because the CPU is no more the bottleneck. In >> a fact now the memory hierarchy has become the new bottleneck (which >> means that using more memory for the same >> tends to be much slower). > > Operating on 16-bit values using natural word-size operations (32 bit > or 64 bit) is never more expensive. And the memory is only ever > 16-bit. So your performance argument does not stand. > > I hope this clarifies your understanding of the Modula-3 type system. > >> >> Regards, >> Elmar >> >>> Can't do this. Non-open arrays must have static constant bounds, >>> which the above does not. Open arrays have extra dope and are accessed >>> through two extra pointers, which we really wouldn't want for text >>> literals. >>> >>> TextLiteral.Ts are unsafe only in UNSAFE code, which is no worse >>> that anything >>> else in UNSAFE code. The interface is UNSAFE, which means safe code >>> can't IMPORT >>> it, and thus can't see the declaration. Not very likely that >>> anybody other than >>> the runtime, compiler-generated code, and debugger (written in C >>> anyway) would >>> want to mess with it. >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Fri Jun 5 10:24:32 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:24:32 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> Message-ID: <55715CC0.401@elstel.org> Am 04.06.15 um 20:32 schrieb Darko Volaric: > You really should try writing your own language, it's really not that > hard. Darko, you are dreaming. Writing a small demo compiler may be one weeks job. However writing something that implements all that your hart desires will be very hard to do if at all doable for a single person. > M3 will never give you satisfaction. You've already implemented a C > back end and no doubt have an idea of how a compiler works. > > As for language features, I think we should be looking for features to > remove. > Jay, with a little more time I would have been at your side in a first place! A couple of years ago I had some own plans for a M3/Pascal like language with similar or even more venturous properties. 1) operator overloading -- C++ 2) templates with type deduction (i.e. C++ function templates, not just class templates or generic modules) -- C++ 3) a small easy to understand language -- not C++, maybe Modula-3 4) a small easy to understand compiler (m3front I don't understand) -- nowhere 5) fast compilation -- Modula-3 6) optional safey -- Modula-3 7) multiple inheritance, at least of "interfaces" -- C++ 8) optionally stack or inline allocation of "classes"/"objects" -- C++ 9) clear choice of build system -- Modula-3 perhaps 10) locals with destructors -- C++ 11) clear portable choice for remoting/RPC -- Modula-3 perhaps 12) needs backend work -- Modula-3 perhaps 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta 14) ?safety w/o garbage collection? -- rust?? 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. Not Python/Perl/Ruby/Erlang/sh/Tcl. 16) static compilation to native code, maybe -- C++ and Modula-3! Go? Net.Native? 17) non-virtual member functions -- not C or Modula-3 18) virtual member functions -- C++, Modula-3 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ was late here. 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 and C are decades late. 22) good string library 23) good regex library Nonetheless I wanna see what can be done with Modula-3 in its current state. - and I believe it already offers some interesting properties compared to C like f.i. range checking and better type safety which could make a difference for security relevant projects like f.i. a web browser. The only thing that would be necessary for such a project was a *** well tested and ready-to-use shipped *** GUI toolkit ... From estellnb at elstel.org Fri Jun 5 10:33:54 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:33:54 +0200 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55715CC0.401@elstel.org> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> <55715CC0.401@elstel.org> Message-ID: <55715EF2.7020500@elstel.org> > 17) non-virtual member functions -- not C or Modula-3 That already works for Modula-3. Simply put the method name with the same signature another time under "methods" instead of overrides (However you would have to do that in each subclass - and there is likely no runtime optimization for it; i.e. it may effectively just declare another virtual method with the same name.). From estellnb at elstel.org Fri Jun 5 10:29:24 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 05 Jun 2015 10:29:24 +0200 Subject: [M3devel] User interfaces for Modula 3. In-Reply-To: <556BD24D.7010902@lcwb.coop> References: <20150601002944.GA10751@topoi.pooq.com> <556BD24D.7010902@lcwb.coop> Message-ID: <55715DE4.3000304@elstel.org> Am 01.06.15 um 05:32 schrieb Rodney M. Bates: > There is a QT binding in cm3/m3-ui/qt. I have no experience > with it. > Dear Hendrik, I am also very interested in a good GUI environment for Modula-3. It would be very nice if you could share your experience with the current Qt bindings if you should resolve to give them a test. Once I had started to port the program at http://www.elstel.org/coan/ to Modula-3. That time there were no Qt-bindings and I had started to program my own bindings for M3. As far as I can remember they use virtual methods instead of signals or slots. I am very likely to continue my work on it at least if the current Qt toolkit should not be as good as desired. By the end of this summer I would suppose CoAn to be available under an OSSy license and being ported to Modula-3. Elmar > On 05/31/2015 07:29 PM, Hendrik Boom wrote: >> What user interface libraries are available for Modula 3? >> >> I know there's Trestle. >> >> But is there something like GTK or QT or somethng I can use like cairo >> to draw really pretty text? Anything that supports lots of Unicode? >> I don't mind havein to do my own character placement based on font >> metrics of some sort; not now, bit later, I may have some really >> weird layout constraints. >> >> In case you're wodering about the application: >> >> I'm doing preliminary planning for something like a text editor with >> several windows, one with the raw text, and another with a continuous >> (as continuous as I have cpu cycles for) display of the results of >> rather >> complex calculations on that text (such as proof checking or >> described graphics). >> >> Modula 3 isn't the only candidate for a programming language, just the >> leading candidate for historical and bootstrapping reasons. The others >> at the moment are OCAML and some kind of statically typed Scheme. >> >> And of course, I may decide that theo whole projecct is infeasible. >> >> -- hendrik >> >> > From adacore at marino.st Fri Jun 5 11:14:50 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 11:14:50 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> Message-ID: <5571688A.4060404@marino.st> On 6/5/2015 09:26, Olaf Wagner wrote: > On Fri, 05 Jun 2015 08:10:08 +0200 > John Marino wrote: > >> The other issue is that there were numerous files in pkg/ that were >> executable and shouldn't have been. I used find/chmod to remove exec >> perms for all except "cm3" file. I did this 1.5 years ago, I'm now >> wondering how right it was. The current list: >> >> pkg/m3core/src/C/Common/Csetjmp.i3 >> pkg/m3back/src/M3C.i3 >> pkg/m3staloneback/AMD64_FREEBSD/m3back >> pkg/cm3/AMD64_FREEBSD/cm3 >> pkg/libdump/AMD64_FREEBSD/libdump >> pkg/cmpfp/AMD64_FREEBSD/cmpfp >> pkg/formsview/AMD64_FREEBSD/formsview >> pkg/vorun/AMD64_FREEBSD/vorun >> pkg/pkl-fonts/AMD64_FREEBSD/PklFonts >> pkg/hack/AMD64_FREEBSD/dummy >> pkg/test/AMD64_FREEBSD/test >> >> Should there be executables in pkg/ ? Is the chmod command then wrong? >> I had to strips these as well to pass QA tests. > > Except for the i3-files these are all programs and should be executable. Okay, when I fixed this internally, I discovered something: m3back, libdump, cmpfp, formsview, vorun, dummy, test all link to m3 libraries but the rpath of $ORIGIN:$ORIGIN/../lib isn't enough. All of these programs need an additional rpath to $ORIGIN/../../../lib for the linker to find the libs. These all need to be fixed in repository I suspect. John From lists at darko.org Fri Jun 5 13:07:20 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 5 Jun 2015 04:07:20 -0700 Subject: [M3devel] 32bit host 64bit target TextLiteral recurring problem In-Reply-To: <55715CC0.401@elstel.org> References: <255891E4-66EE-4220-9FA5-0D7622D05B4D@gmail.com> <395BA1D2-E396-4A0B-B4E9-EA72018B4EDD@purdue.edu> <3BE59589-F337-45F1-BFCF-3DDA0FD45F53@purdue.edu> <55706292.9060608@lcwb.coop> <7156B516-49FB-4504-8319-95670AAF1413@purdue.edu> <55715CC0.401@elstel.org> Message-ID: I guess it depends on your level of skills, but source to source compilers aren't that challenging. The more important point is that exercise might focus Jay's mind on the the complexity and desirability of those features. It's all fine talking about them in the abstract, it's completely different when you get down to brass tacks. On Fri, Jun 5, 2015 at 1:24 AM, Elmar Stellnberger wrote: > Am 04.06.15 um 20:32 schrieb Darko Volaric: > >> You really should try writing your own language, it's really not that >> hard. >> > Darko, you are dreaming. Writing a small demo compiler may be one weeks > job. > However writing something that implements all that your hart desires will > be > very hard to do if at all doable for a single person. > > M3 will never give you satisfaction. You've already implemented a C back >> end and no doubt have an idea of how a compiler works. >> >> As for language features, I think we should be looking for features to >> remove. >> >> Jay, with a little more time I would have been at your side in a first > place! > A couple of years ago I had some own plans for a M3/Pascal like language > with similar or even more venturous properties. > > 1) operator overloading -- C++ > 2) templates with type deduction (i.e. C++ function templates, not just > class templates or generic modules) -- C++ > 3) a small easy to understand language -- not C++, maybe Modula-3 > 4) a small easy to understand compiler (m3front I don't understand) -- > nowhere > 5) fast compilation -- Modula-3 > 6) optional safey -- Modula-3 > 7) multiple inheritance, at least of "interfaces" -- C++ > 8) optionally stack or inline allocation of "classes"/"objects" -- C++ > 9) clear choice of build system -- Modula-3 perhaps > 10) locals with destructors -- C++ > 11) clear portable choice for remoting/RPC -- Modula-3 perhaps > 12) needs backend work -- Modula-3 perhaps > 13) a small easty to understand backend -- Modula-3 perhaps kinda sorta > 14) ?safety w/o garbage collection? -- rust?? > 15) static compilation for type checking -- C++, Modula-3, Java, Go, C#. > Not Python/Perl/Ruby/Erlang/sh/Tcl. > 16) static compilation to native code, maybe -- C++ and Modula-3! Go? > Net.Native? > 17) non-virtual member functions -- not C or Modula-3 > 18) virtual member functions -- C++, Modula-3 > 19) safe numbers? Scheme?? C++ w/ library? (with operator overloading) > 20) clear/portable choice for threads -- Modula-3, C++14? Java. Geez C++ > was late here. > 21) clear/portable choice for interlocked -- Win32, msvc gcc; Modula-3 > and C are decades late. > 22) good string library > 23) good regex library > > Nonetheless I wanna see what can be done with Modula-3 in its current > state. - > and I believe it already offers some interesting properties compared to C > like > f.i. range checking and better type safety which could make a difference > for > security relevant projects like f.i. a web browser. The only thing that > would be > necessary for such a project was a *** well tested and ready-to-use > shipped *** > GUI toolkit ... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Jun 5 17:42:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 05 Jun 2015 08:42:17 -0700 Subject: [M3devel] rpath woes In-Reply-To: References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>, <5570D114.40104@marino.st> Message-ID: <20150605154217.AABC81A2062@async.async.caltech.edu> Jay K writes: >--_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > The FreeBSD4 users were surprisingly vocal surprisingly recently. > So I put some work into it. > I agree it is an old system. I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, but the 4.x config worked well. I've since upgraded to 10.0-RELEASE. And don't use it much anymore because posix threads (still) don't work right on FreeBSD. I spent a considerable effort to get them working right on Debian, which they now do, but didn't have the time to figure it out for a bunch of other OSes as well. It's really pretty nice to have them working correctly. They aren't perfect (a bit too much locking over garbage collection issues) but they don't ever seem to crash. On problems with lots of parallelism you do get some parallel speedup, even. Mika From jay.krell at cornell.edu Fri Jun 5 20:51:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 5 Jun 2015 18:51:13 +0000 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <5571688A.4060404@marino.st> References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st> Message-ID: This is kind of on purpose as Olaf said and kind of broken as you point out. We often build_standalone() to workaround the origin/rpath matte. We kind of fail here, in terms of library paths. I don't think the original system fully accounted for runpath. I think at the time, LD_LIBRARY_PATH might have been considered an acceptable solution. Elsewhere, with libtool, make install relinks to reset paths. We don't have that implemented. It is somewhat a good solution. - Jay > Date: Fri, 5 Jun 2015 11:14:50 +0200 > From: adacore at marino.st > To: wagner at elegosoft.com > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] install issues (stripping, permissions) > > On 6/5/2015 09:26, Olaf Wagner wrote: > > On Fri, 05 Jun 2015 08:10:08 +0200 > > John Marino wrote: > > > >> The other issue is that there were numerous files in pkg/ that were > >> executable and shouldn't have been. I used find/chmod to remove exec > >> perms for all except "cm3" file. I did this 1.5 years ago, I'm now > >> wondering how right it was. The current list: > >> > >> pkg/m3core/src/C/Common/Csetjmp.i3 > >> pkg/m3back/src/M3C.i3 > >> pkg/m3staloneback/AMD64_FREEBSD/m3back > >> pkg/cm3/AMD64_FREEBSD/cm3 > >> pkg/libdump/AMD64_FREEBSD/libdump > >> pkg/cmpfp/AMD64_FREEBSD/cmpfp > >> pkg/formsview/AMD64_FREEBSD/formsview > >> pkg/vorun/AMD64_FREEBSD/vorun > >> pkg/pkl-fonts/AMD64_FREEBSD/PklFonts > >> pkg/hack/AMD64_FREEBSD/dummy > >> pkg/test/AMD64_FREEBSD/test > >> > >> Should there be executables in pkg/ ? Is the chmod command then wrong? > >> I had to strips these as well to pass QA tests. > > > > Except for the i3-files these are all programs and should be executable. > > Okay, when I fixed this internally, I discovered something: > m3back, libdump, cmpfp, formsview, vorun, dummy, test all link to m3 > libraries but the rpath of $ORIGIN:$ORIGIN/../lib isn't enough. All of > these programs need an additional rpath to $ORIGIN/../../../lib for the > linker to find the libs. These all need to be fixed in repository I > suspect. > > John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jun 5 22:53:14 2015 From: adacore at marino.st (John Marino) Date: Fri, 05 Jun 2015 22:53:14 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st> Message-ID: <55720C3A.5070200@marino.st> On 6/5/2015 20:51, Jay K wrote: > This is kind of on purpose as Olaf said and kind of broken as you point out. > > We often build_standalone() to workaround the origin/rpath matte. > > We kind of fail here, in terms of library paths. well, to be blunt, it -is- broken and thus is a fail. > I don't think the original system fully accounted for runpath. > I think at the time, LD_LIBRARY_PATH might have been considered an > acceptable solution. If I wanted to go down a path like this, I'd use ldconfig, but my personal feeling if is ldconfig is required, the software is doing something wrong. > Elsewhere, with libtool, make install relinks to reset paths. We don't > have that implemented. > It is somewhat a good solution. I think it has to be a tweak at FreeBSD.common -- rather than use $ORIGIN/../lib for rpath, it needs a conditional statement to use $ORIGIN/../../../lib instead. I just don't know what the condition is. I'm also trying to figure out why rpath=$ORIGIN is needed. are there ever libraries in the same directory as cm3? ohn From jay.krell at cornell.edu Sat Jun 6 00:59:55 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 15:59:55 -0700 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55720C3A.5070200@marino.st> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> Message-ID: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> We could use origin, origin/../lib, origin/../../lib etc? But eventually a security problem to reach all around? Plain origin is in case the files are all together in one directory like on windows. Isn't it desirable to be buildable w/ system cc/ld and latest? Thank you for this work. - Jay On Jun 5, 2015, at 1:53 PM, John Marino wrote: > On 6/5/2015 20:51, Jay K wrote: >> This is kind of on purpose as Olaf said and kind of broken as you point out. >> >> We often build_standalone() to workaround the origin/rpath matte. >> >> We kind of fail here, in terms of library paths. > > well, to be blunt, it -is- broken and thus is a fail. > >> I don't think the original system fully accounted for runpath. >> I think at the time, LD_LIBRARY_PATH might have been considered an >> acceptable solution. > > If I wanted to go down a path like this, I'd use ldconfig, but my > personal feeling if is ldconfig is required, the software is doing > something wrong. > > >> Elsewhere, with libtool, make install relinks to reset paths. We don't >> have that implemented. >> It is somewhat a good solution. > > I think it has to be a tweak at FreeBSD.common -- rather than use > $ORIGIN/../lib for rpath, it needs a conditional statement to use > $ORIGIN/../../../lib instead. I just don't know what the condition is. > > I'm also trying to figure out why rpath=$ORIGIN is needed. are there > ever libraries in the same directory as cm3? > > ohn From jay.krell at cornell.edu Sat Jun 6 01:01:48 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 16:01:48 -0700 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> Message-ID: <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> Also "dist" means "distribution" means run on other machines, try not to be too specific to build machine. That is why not full paths. - Jay On Jun 5, 2015, at 3:59 PM, Jay wrote: > We could use origin, origin/../lib, origin/../../lib etc? But eventually a security problem to reach all around? Plain origin is in case the files are all together in one directory like on windows. > > Isn't it desirable to be buildable w/ system cc/ld and latest? > > Thank you for this work. > > - Jay > > On Jun 5, 2015, at 1:53 PM, John Marino wrote: > >> On 6/5/2015 20:51, Jay K wrote: >>> This is kind of on purpose as Olaf said and kind of broken as you point out. >>> >>> We often build_standalone() to workaround the origin/rpath matte. >>> >>> We kind of fail here, in terms of library paths. >> >> well, to be blunt, it -is- broken and thus is a fail. >> >>> I don't think the original system fully accounted for runpath. >>> I think at the time, LD_LIBRARY_PATH might have been considered an >>> acceptable solution. >> >> If I wanted to go down a path like this, I'd use ldconfig, but my >> personal feeling if is ldconfig is required, the software is doing >> something wrong. >> >> >>> Elsewhere, with libtool, make install relinks to reset paths. We don't >>> have that implemented. >>> It is somewhat a good solution. >> >> I think it has to be a tweak at FreeBSD.common -- rather than use >> $ORIGIN/../lib for rpath, it needs a conditional statement to use >> $ORIGIN/../../../lib instead. I just don't know what the condition is. >> >> I'm also trying to figure out why rpath=$ORIGIN is needed. are there >> ever libraries in the same directory as cm3? >> >> ohn From jay.krell at cornell.edu Sat Jun 6 02:20:44 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 5 Jun 2015 17:20:44 -0700 Subject: [M3devel] rpath woes In-Reply-To: <20150605154217.AABC81A2062@async.async.caltech.edu> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> <5570D114.40104@marino.st> <20150605154217.AABC81A2062@async.async.caltech.edu> Message-ID: <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> Posix threads from C? From Modula-3? - Jay On Jun 5, 2015, at 8:42 AM, wrote: > Jay K writes: >> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> The FreeBSD4 users were surprisingly vocal surprisingly recently. >> So I put some work into it. >> I agree it is an old system. > > I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, > but the 4.x config worked well. > > I've since upgraded to 10.0-RELEASE. And don't use it much anymore > because posix threads (still) don't work right on FreeBSD. I spent a > considerable effort to get them working right on Debian, which they > now do, but didn't have the time to figure it out for a bunch of > other OSes as well. It's really pretty nice to have them working > correctly. They aren't perfect (a bit too much locking over garbage > collection issues) but they don't ever seem to crash. On problems > with lots of parallelism you do get some parallel speedup, even. > > Mika From mika at async.caltech.edu Sat Jun 6 02:32:10 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 05 Jun 2015 17:32:10 -0700 Subject: [M3devel] rpath woes In-Reply-To: <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> References: <55709183.4060804@marino.st> <5570947F.2080600@marino.st> <5570D114.40104@marino.st> <20150605154217.AABC81A2062@async.async.caltech.edu> <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com> Message-ID: <20150606003210.50F2E1A2065@async.async.caltech.edu> I think I've mentioned this on this mailing list about a hundred times, but... the pthreads implementation of Modula-3 threads doesn't work on most OSes. As far as I know the only OS where it is truly reliably working is AMD64_LINUX. I certainly haven't tested all targets. User-level threads work everywhere. Except for AMD64_LINUX I would not suggest putting anything important to use Modula-3 threads built on pthreads. They work... until they don't. Anyone wishing to work on this and fix it, please don't change the AMD64_LINUX code at all unless you know exactly, precisely what you are doing. It is very very difficult to debug the subtle bugs that get introduced. As far as I know there's no problem with C pthreads on FreeBSD. Mika Jay writes: >Posix threads from C? From Modula-3? > > - Jay > >On Jun 5, 2015, at 8:42 AM, wrote: > >> Jay K writes: >>> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ >>> Content-Type: text/plain; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> The FreeBSD4 users were surprisingly vocal surprisingly recently. >>> So I put some work into it. >>> I agree it is an old system. >> >> I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, >> but the 4.x config worked well. >> >> I've since upgraded to 10.0-RELEASE. And don't use it much anymore >> because posix threads (still) don't work right on FreeBSD. I spent a >> considerable effort to get them working right on Debian, which they >> now do, but didn't have the time to figure it out for a bunch of >> other OSes as well. It's really pretty nice to have them working >> correctly. They aren't perfect (a bit too much locking over garbage >> collection issues) but they don't ever seem to crash. On problems >> with lots of parallelism you do get some parallel speedup, even. >> >> Mika From adacore at marino.st Sat Jun 6 07:53:40 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 07:53:40 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> <6CE705F1-7518-4D0D-8490-C9B2625C7343@gmail.com> Message-ID: <55728AE4.4030301@marino.st> On 6/6/2015 01:01, Jay wrote: > Also "dist" means "distribution" means run on other machines, try not to be too specific to build machine. That is why not full paths. > > - Jay > > On Jun 5, 2015, at 3:59 PM, Jay wrote: >>> I think it has to be a tweak at FreeBSD.common -- rather than use >>> $ORIGIN/../lib for rpath, it needs a conditional statement to use >>> $ORIGIN/../../../lib instead. I just don't know what the condition is. >>> >>> I'm also trying to figure out why rpath=$ORIGIN is needed. are there >>> ever libraries in the same directory as cm3? >>> >>> ohn I think you misunderstood. I know why "$ORIGIN/../lib" is there. I don't know why "$ORIGIN" is there. There are no libraries in /usr/local/cm3/bin, so why is this directory specified? John From adacore at marino.st Sat Jun 6 07:57:50 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 07:57:50 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> References: <55713D40.6080305@marino.st> <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com> <5571688A.4060404@marino.st> <55720C3A.5070200@marino.st> <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com> Message-ID: <55728BDE.8030702@marino.st> On 6/6/2015 00:59, Jay wrote: > We could use origin, origin/../lib, origin/../../lib etc? But > eventually a security problem to reach all around? Plain origin is in > case the files are all together in one directory like on windows. I'm not really a fan of just adding every possible rpath. In this case, it's either $ORIGIN/../lib or $ORIGIN/../../../lib, but together is never correct (as the other mail indicates, $ORIGIN by itself also seems to be always wrong) > > Isn't it desirable to be buildable w/ system cc/ld and latest? On DragonFly - yes On FreeBSD - no. FreeBSD is stuck with either ancient gcc (4.2.1), ancient binutils (2.17) or clang. In every case, it needs to use newer GCC and binutils from ports to build M3. (and every newer GCC) John From jay.krell at cornell.edu Sat Jun 6 10:47:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 6 Jun 2015 08:47:25 +0000 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: <55728BDE.8030702@marino.st> References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st>, , <55720C3A.5070200@marino.st>, <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com>, <55728BDE.8030702@marino.st> Message-ID: $ORIGIN is "in case" we restructure Unix install to look like Windows install -- moving lib on top of bin. I realize you'd like just the minimum required, not hypothetical values for the future. Regarding gcc/ld -- isn't it desirable to have either/both work? Are there command lines that work with both and provide the same meaning? Does our stuff not work with older gcc/ld? Do ports in general avoid system gcc/ld? Does clang work? I intend to try. - Jay > Date: Sat, 6 Jun 2015 07:57:50 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] install issues (stripping, permissions) > > On 6/6/2015 00:59, Jay wrote: > > We could use origin, origin/../lib, origin/../../lib etc? But > > eventually a security problem to reach all around? Plain origin is in > > case the files are all together in one directory like on windows. > > I'm not really a fan of just adding every possible rpath. In this case, > it's either $ORIGIN/../lib or $ORIGIN/../../../lib, but together is > never correct (as the other mail indicates, $ORIGIN by itself also seems > to be always wrong) > > > > > Isn't it desirable to be buildable w/ system cc/ld and latest? > > On DragonFly - yes > On FreeBSD - no. > > FreeBSD is stuck with either ancient gcc (4.2.1), ancient binutils > (2.17) or clang. In every case, it needs to use newer GCC and binutils > from ports to build M3. (and every newer GCC) > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jun 6 10:48:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 6 Jun 2015 08:48:41 +0000 Subject: [M3devel] rpath woes In-Reply-To: <20150606003210.50F2E1A2065@async.async.caltech.edu> References: <55709183.4060804@marino.st>, <5570947F.2080600@marino.st>,<5570D114.40104@marino.st> , <20150605154217.AABC81A2062@async.async.caltech.edu>, <5395D24D-F343-4156-AF98-0FDD33BB0415@gmail.com>, <20150606003210.50F2E1A2065@async.async.caltech.edu> Message-ID: I thought these problems were all fixed by now. :( Maybe cooperative suspend will help. I know that seems like a non-sequiter, but I know that SuspendThread and GetThreadContext on Windows don't do what you'd expect and as a result our x86-on-amd64 code isn't correct. - Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > Date: Fri, 5 Jun 2015 17:32:10 -0700 > From: mika at async.caltech.edu > > I think I've mentioned this on this mailing list about a hundred times, but... > > the pthreads implementation of Modula-3 threads doesn't work on most OSes. > > As far as I know the only OS where it is truly reliably working is AMD64_LINUX. > > I certainly haven't tested all targets. > > User-level threads work everywhere. > > Except for AMD64_LINUX I would not suggest putting anything important to > use Modula-3 threads built on pthreads. They work... until they don't. > > Anyone wishing to work on this and fix it, please don't change the > AMD64_LINUX code at all unless you know exactly, precisely what you > are doing. It is very very difficult to debug the subtle bugs that > get introduced. > > As far as I know there's no problem with C pthreads on FreeBSD. > > Mika > > Jay writes: > >Posix threads from C? From Modula-3? > > > > - Jay > > > >On Jun 5, 2015, at 8:42 AM, wrote: > > > >> Jay K writes: > >>> --_da40b763-6cc9-48e7-9a39-2f2565af44c7_ > >>> Content-Type: text/plain; charset="iso-8859-1" > >>> Content-Transfer-Encoding: quoted-printable > >>> > >>> The FreeBSD4 users were surprisingly vocal surprisingly recently. > >>> So I put some work into it. > >>> I agree it is an old system. > >> > >> I think you mean "the FreeBSD4 user"... I was actually running FreeBSD 5, > >> but the 4.x config worked well. > >> > >> I've since upgraded to 10.0-RELEASE. And don't use it much anymore > >> because posix threads (still) don't work right on FreeBSD. I spent a > >> considerable effort to get them working right on Debian, which they > >> now do, but didn't have the time to figure it out for a bunch of > >> other OSes as well. It's really pretty nice to have them working > >> correctly. They aren't perfect (a bit too much locking over garbage > >> collection issues) but they don't ever seem to crash. On problems > >> with lots of parallelism you do get some parallel speedup, even. > >> > >> Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sat Jun 6 14:02:29 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 14:02:29 +0200 Subject: [M3devel] install issues (stripping, permissions) In-Reply-To: References: <55713D40.6080305@marino.st>, <20150605092618.b05203ee6831ed73f7d5b8e0@elegosoft.com>, <5571688A.4060404@marino.st>, , <55720C3A.5070200@marino.st>, <22A00DAA-F814-41D6-BB36-E4F0491B1917@gmail.com>, <55728BDE.8030702@marino.st> Message-ID: <5572E155.2040408@marino.st> On 6/6/2015 10:47, Jay K wrote: > $ORIGIN is "in case" we restructure Unix install to look like Windows > install -- moving lib on top of bin. > I realize you'd like just the minimum required, not hypothetical values > for the future. Since it's patched anyway, I'll just remove $ORIGIN then. Obviously I'm going to vote against a lowest-common-denomination approach of dumping everything into one directory. A better way is have the build system support either. > Regarding gcc/ld -- isn't it desirable to have either/both work? Given scarce developer resources (and small user population), I recommend that you target supporting ports rather than try to support multiple configurations. Unless you are really committed to supported platforms that have been EOL for years (basically FreeBSD 8.x and lower) then I don't see a big ROI on supporting more than what ports requires. To specifically answer, I'd say "no". FreeBSD 9, 10, 11 all have different base compiler configurations. It's not worth trying to support the quirks of each one. > Are there command lines that work with both and provide the same meaning? I'm not sure, but since ports works on FreeBSD 9 and later, I am not that motivated to do the extensive testing necessary to answer that. > Does our stuff not work with older gcc/ld? > Do ports in general avoid system gcc/ld? More and more. The freebsd base assembler is ancient, so often a new gas is required which brings in the new ld and forces that ports GCC be used (if not base clang) > Does clang work? > I intend to try. I expect it to work on the C-backend. John From mika at async.caltech.edu Sat Jun 6 18:50:23 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 06 Jun 2015 09:50:23 -0700 Subject: [M3devel] the need for cooperative suspend In-Reply-To: References: <20150502155713.29DCC1A2062@async.async.caltech.edu> Message-ID: <20150606165023.78AEC1A2066@async.async.caltech.edu> Can the thread tester survive with the most paranoid options on OS X? I don't believe it until I've seen it (based on what we went through for Linux, I would say I'm just being prudent). Run it with: "-iters 100000 -n 20 -tests ALL" I remember the thread tester died on FreeBSD last time I tried, and there have been no relevant checkins since that I know of. Here's what happens when I run it on my FreeBSD. CM3 is not up to date, in case someone fixed something I'm not aware of, so don't draw too many conclusions until someone has tried it on a fresh install: ..........laziest thread is 1433608799/9/1 (tests: read 10/6/6 nxread 26/6/4 tryexcept 67/53/38 fork 1433608799/2/2 forktoomuch 1433608799/9/3 alloc 1433608799/37/1 creat 10/7/7 lock 1433608799/68/38) *** *** runtime error: *** Segmentation violation - possible attempt to dereference NIL *** pc = 0x4387e8 = Move + 0x6a in ../src/runtime/common/RTCollector.m3 *** Abort (79)pluto:/tmp> Anyone who's interested in trying their own favorite Modula-3 installation, this is easy to do: cm3/m3-libs/m3core/tests/thread/AMD64_FREEBSD/threadtest -iters 100000 -n 20 -tests ALL Ignore the warnings about "thread being starved or deadlocked". Those should really be an option and not on by default: as far as I know the threading library makes no claim of fairness, so I don't think one should expect it by default. Mika Antony Hosking writes: >OS X is stable as well as Linux, as far as I know. >Mika, what misbehaviors are you seeing? > >Sent from my iPad > >> On May 2, 2015, at 11:57 AM, mika at async.caltech.edu wrote: >>=20 >> Jay,=20 >>=20 >> Can you explain precisely what you mean by "suspend"? >>=20 >> A thread is suspended by ... a timer expiration? Another thread at higher= > priority needing resources? >>=20 >> I'm thinking about a tight loop "for(;;) ;"... how do you stop a thread fr= >om running that (it could have >> been coded in assembly...)? >>=20 >> Jay K writes: >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_ >>> Content-Type: text/plain; charset=3D"iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>>=20 >>> We really need to move away from preemptive suspend. >>>=20 >>> 1) It is a large fraction of the target-dependent code in m3core. >>> =3D20 >>>=20 >>> 2) It doesn't really work in "wow64"=3D2C the x86-on-amd64 Windows system= >. >>> The SuspendThread + GetThreadContext does not actually reliably give you >>> current and coherent context. >>> I'm not sure about x86-on-ia64. >>> =3D20 >>>=20 >>> The context can be old. >>> The context can be mid-update. Which might be ok=3D2C depending on what t= >he =3D >>> updates are. >>> LIke if Eip and Esp are being updated=3D2C we don't care about Eip anyway= >. >>> =3D20 >>>=20 >>> You can find out if either is the case=3D2C and let the thread run longer= >=3D2C >>> but being in a long running syscall I believe is reported as possibly >>> invalid even though it is valid. >>> =3D20 >>> =3D20 >>> NT/native is ok. >>> =3D20 >>>=20 >>> We should just use cooperative suspend and not worry about it. >>> =3D20 >>> =3D20 >>> Thanks=3D2C >>> - Jay >>>=20 >>>=20 >>> =3D20 >>> =3D >>>=20 >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_ >>> Content-Type: text/html; charset=3D"iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>>=20 >>> >>> >>> >>>
We really need to move awa= >y from=3D >>> preemptive suspend.

1) It is a large fraction of the target-depend= >e=3D >>> nt code in m3core.
 =3D3B

2) It doesn't really work in "wow= >64"=3D >>> =3D2C the x86-on-amd64 Windows system.
The SuspendThread + GetThreadCo= >ntex=3D >>> t does not actually reliably give you
current and coherent context.>I=3D >>> 'm not sure about x86-on-ia64.
 =3D3B

 =3D3BThe context= > can b=3D >>> e old.
 =3D3BThe context can be mid-update. Which might be ok=3D2C= > depen=3D >>> ding on what the updates are.
 =3D3BLIke if Eip and Esp are being u= >pda=3D >>> ted=3D2C we don't care about Eip anyway.
 =3D3B

You can fin= >d out =3D >>> if either is the case=3D2C and let the thread run longer=3D2C
but bein= >g in a=3D >>> long running syscall I believe is reported as possibly
invalid even th= >o=3D >>> ugh it is valid.
 =3D3B
 =3D3B
NT/native is ok.
 = >=3D3B>>>
We should just use cooperative suspend and not worry about it.
&n= >bs=3D >>> p=3D3B
 =3D3B
Thanks=3D2C
 =3D3B- Jay


 =3D= >3B
=3D >>>
>>> =3D >>>=20 >>> --_6bc62225-3263-437a-a88c-4b1ff9473d58_-- From adacore at marino.st Sat Jun 6 23:32:52 2015 From: adacore at marino.st (John Marino) Date: Sat, 06 Jun 2015 23:32:52 +0200 Subject: [M3devel] Follow up - FreeBSD port Message-ID: <55736704.6030900@marino.st> Based on recent discussions, I altered the FreeBSD port as follows: 1) I removed the rpath=$ORIGIN flag 2) I limited the de-execution to the executable .i3 files 3) I moved the executable pkg programs to bin, then added softlinks to where the were originally installed. The last one enables those programs to work again, even from the pkg/*/AMD64_FREEBSD/ locations. I could not figure out a way to dynamically change the rpath so I just decided to use $ORIGIN/../lib for everything an make it work. It has a side benefit of eliminating the large, duplicate cm3 at pkg/cm3/AMD64_FREEBSD/cm3 (replaced by symlink). I think the port is in pretty good shape right now. Most of the programs work, but cm3ide core dumps a lot. It also seems to be querying a database that doesn't exist (pgsql installed but port doesn't do anything with it). Anything that tries to use TCP seems to fail. John http://www.freshports.org/commit.php?category=lang&port=modula3&files=yes&message_id=201506062130.t56LUF55083786 at svn.freebsd.org From mika at async.caltech.edu Sun Jun 7 08:05:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 06 Jun 2015 23:05:17 -0700 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <55736704.6030900@marino.st> References: <55736704.6030900@marino.st> Message-ID: <20150607060517.0FF6E1A2067@async.async.caltech.edu> John Marino writes: ... >Anything that tries to use TCP seems to fail. ... Might be a wild goose chase.. but I seem to remember having issues if the reverse and forward DNS of your machine don't match up, or if there is no reverse DNS... or something like that. Otherwise, TCP should work just fine. Mika From hendrik at topoi.pooq.com Sun Jun 7 14:47:12 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 7 Jun 2015 08:47:12 -0400 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <20150607060517.0FF6E1A2067@async.async.caltech.edu> References: <55736704.6030900@marino.st> <20150607060517.0FF6E1A2067@async.async.caltech.edu> Message-ID: <20150607124712.GA1045@topoi.pooq.com> On Sat, Jun 06, 2015 at 11:05:17PM -0700, mika at async.caltech.edu wrote: > John Marino writes: > ... > >Anything that tries to use TCP seems to fail. > ... > > Might be a wild goose chase.. but I seem to remember having issues if > the reverse and forward DNS of your machine don't match up, or if there > is no reverse DNS... or something like that. That's not TCP doing that as far as I know. Even DNS is built on top of TCP and IP, and TCP knows nothing of it. However, a lot of email servers perform such a check to block some forms of malfeasance. Possibly some other systems built on top of TCP do that as well. > > Otherwise, TCP should work just fine. > > Mika From mika at async.caltech.edu Sun Jun 7 18:03:50 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 07 Jun 2015 09:03:50 -0700 Subject: [M3devel] Follow up - FreeBSD port In-Reply-To: <20150607124712.GA1045@topoi.pooq.com> References: <55736704.6030900@marino.st> <20150607060517.0FF6E1A2067@async.async.caltech.edu> <20150607124712.GA1045@topoi.pooq.com> Message-ID: <20150607160350.D5C1F1A2066@async.async.caltech.edu> Hendrik Boom writes: >On Sat, Jun 06, 2015 at 11:05:17PM -0700, mika at async.caltech.edu wrote: >> John Marino writes: >> ... >> >Anything that tries to use TCP seems to fail. >> ... >> >> Might be a wild goose chase.. but I seem to remember having issues if >> the reverse and forward DNS of your machine don't match up, or if there >> is no reverse DNS... or something like that. > >That's not TCP doing that as far as I know. Even DNS is built >on top of TCP and IP, and TCP knows nothing of it. > >However, a lot of email servers perform such a check to block some >forms of malfeasance. Possibly some other systems built on top of >TCP do that as well. > >> >> Otherwise, TCP should work just fine. >> >> Mika Well, yeah, it's obviously not TCP itself---don't think anyone claimed as much. It's something about the Modula-3 code that wraps TCP socket code making assumptions about the configuration of the host it's running on (that that host is configured in a "reasonable way" for a workstation in an industrial research lab, perhaps...) I remember running into it a few times and don't know if I (or anyone) ever fixed it in the distribution. Mika From rodney_bates at lcwb.coop Sun Jun 7 19:32:59 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 07 Jun 2015 12:32:59 -0500 Subject: [M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent? In-Reply-To: <55708E2B.4040607@elstel.org> References: <55705A41.7070503@elstel.org> <55707E3B.7020601@lcwb.coop> <55708E2B.4040607@elstel.org> Message-ID: <5574804B.5030304@lcwb.coop> On 06/04/2015 12:43 PM, Elmar Stellnberger wrote: > Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral recurring problem > > Am 04.06.15 um 18:35 schrieb Rodney M. Bates: >> >> >> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote: >>> Am 03.06.15 um 03:58 schrieb Jay K: >>>> We cannot cross from 32bit host to 64bit target. >>>> >>>> >>>> Ok to commit this? >>>> >>>> >>> >>> Compatibility between the 32bit and the 64bit version of the same program would be the minimum I expected from a pickle. >>> To me this is just another argument for a language specified value range of INTEGER. >>> It makes certain things easier. >>> >>> My suggestion was: >>> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 32 FOR INTEGER for 32 and 64bit targets) >>> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I believe there is no reason to forbid it for INTEGER) >>> BITS 64 FOR INTEGER would only work on x64 systems. >>> >> >> What you want is already pretty much here. Just declare your desired type yourself: >> >> TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF]; > This should be good news. However my suggestion was to make Int32.T the default > as this will be somewhat faster for the average application which is data intensive (just > think of image processing) which will never be rewritten to use Int32.T instead of INT. > All of our current and old programs being prepared to run on a 32bit as well as a 64bit > target should compile and run without any problem. There is one single exception > where the compiler would throw an error: when using INTEGER for address arithmetics > in unsafe modules (error is: can only LOOPHOLE between types of the same size, or > some way similar as I have already tested this). There we would need the so called > targetint/offset. This would be a very biased thing to do. The only people who would benefit would be those who know for certain that they want *every* integer variable to have 32-bit range, on both 32-bit and 64-bit machines. Without debating the wisdom or likelihood of this, a quite easy string search/replace on "INTEGER" would accomplish the same thing. But everybody else would have to vet their integers one by one, to see which they wanted always 32 and which they wanted native size. It would break lots of code for the unlucky (majority, I would guess), while providing only speed improvements for the favored. I have worked on lots of code in the M3 distribution itself that would break. > If I understand things right the following two things would need to get implemented > to allow a default int size of 32 for existing applications (be it the default or just an > additional compiler option as the size of WIDECHAR already is): > * automatic range adaption for packed ints (i.e. those with BITS .. FOR) No, your are still trying to misuse BITS to control the value range. Subranges do this just fine. Use the mechanism the language already provides. BITS is for manual control of record/object/array layout only, not value range. > * allowing non 32bit alignment of INTs in memory (as is already the case for WIDE[CHAR]s) INTEGERs (and subranges thereof) are currently no different from WIDECHARS, in regard to alignment. They get the same treatment as anything else. For BITS types inside a record/object/array, the language lets the compiler choose what it will accept, but I am reasonably sure all existing M3 compiler variations, for all targets, will lay things out as you ask (including misaligned) and will correctly access them, as long as no item crosses a native word boundary. Outside records/objects/arrays, the compiler just aligns as needed. >> >> It will always have this range and occupy 32 bits, regardless of the native word size. Pickles >> will always save and reread it with exactly this range, even between machines of different >> word size. > So the target size of the pickle is already determined by the respective INT subrange, right? Defining what pickles do/should do is fraught with pitfalls. Here's what we now have: For (almost) all types, pickles preserve the properties of the M3 type. The bounds of a subrange type are static constant numbers, and are evaluated at compile time on the compiling machine. So [0..16_3FFFFFFF] is the same type on either size of machine, but [0..LAST(INTEGER) DIV 2] is not. On a 32-bit machine, [0..LAST(INTEGER) DIV 2] evaluates to, and thus means [0..16_3FFFFFFF], but on a 64-bit machine, it means [0..16_3FFFFFFFFFFFFFFF], a different type. So, for example, you could exchange pickled values between 32-bit and 64-bit machines if the field/element of the containing record/object/array type were written as [0..16_3FFFFFFF] on both machines, or if it was written as [0..LAST(INTEGER) DIV 2] on the 32-bit machine, and [0..16_3FFFFFFF] on the 64, but not if both had coded the type as [0..LAST(INTEGER) DIV 2]. INTEGER, CARDINAL, LONGINT, and LONGCARD are exceptions. Each of these is treated as the same M3 type, regardless of native word size and upper bound. So pickles will copy a field/element of any these types with a size change, when written and read on different sized machines. The size change does the more-or-less obvious sign-extend one way and raises an exception if the value won't fit, when going the other way. The floating types are not handled by pickles across machines with different floating formats. If the format is the same, (and one pickles knows exists), they will work, otherwise, they will raise an exception. It would be nice to implement this, but apparently, nobody has had a sufficient need. I am no numerical analyst, but I think defining good usable rules for this would call for the services of someone who is. > Something that should to my believe be well documented as one could easily like to extend > an enum with 255 members to more or equal than 256 an then wonder why the new an the > old file formats become incompatible. To circumvent such a problem we would likely need > to allow BITS 8*anything FOR INTEGER and take the bitcount specified in BITS for in order to > determine the target size of an object instead of semi-automatically readjusting it all the > time. This one really gets into a tar pit. Pickles encode the type of a value as a "fingerprint", which is a 64-bit has of the M3 type structure. A copy of this goes into the pickle for every type it contains. Meanwhile, the compiler puts copies of the fingerprints of all types in the program into static global readonly data. Pickle read uses this to look or a type defined in the reading program with the same signature, which means it must be exactly the same type, for the read to succeed. Allowing similar but unequal types would very difficult to define even what differences would be tolerated. This get very complicated very fast. Remember, there is no guarantee that the reading program is the same program as the writing program, or has much similarity to it at all. Pickles requires only that the reading program have a defined reference type that is structurally identical to that of the object had in the writing program. Allowing to write a field of type {Red, Green} in a pickle and read it into a program that has {Red, Green, Blue} would get very messy. Remember that pickles does not handle scalar values, just heap objects. So we would have to be defining some recursive rule about record types whose corresponding field types were each sufficiently "similar", etc, etc. And to implement, the pickle format would have to be completely reworked to contain full type structural descriptions, not just signatures. >> >> The only thing that will differ from 32- to 64-bit machine is the size of intermediate results in >> expressions. If you are already _not_ being careful about avoiding overflows, there are probably >> cases where the results of silent overflows would differ with machine word size. Not sure what >> you would really want here. The unsigned arithmetic operations in Word are explicitly defined >> to silently do binary twos-complement wrap-around when overflows happen. The language does not >> define this for the builtin signed operators. In code where I care about overflows, I never assume >> this. It is probably machine-dependent. > Yes, I believe it would be good like this. No real 32bit INT with 32bit arithmetics but just restricting > the range for Int32.T when it gets written into memory. >> >> Or, you can import it: Int32.T. is already declared, in m3core, with this range. >> >> However, Int32.T specifies BITS 32 FOR ... I strongly suggest this is almost never what you really want. >> The only place the BITS specification makes any difference is when you declare a record or object >> field of this type. In all other situations, BITS 32 FOR .. has _no effect_. The only purpose of >> BITS is so you can force the layout of a record. In that case, since the range needs exactly 32 >> bits anyway, even if you omit the BITS, a field of this type will still always occupy 32 bits. > Nonetheless it is currently not possible to declare a variable of a 32bit type globally: > > TYPE Pixel = BITS 32 FOR INTEGER; > > unluckily does not work as we can have no global variable of type Pixel. No, you have misunderstood. Any type BITS n OF T can be used anywhere T can be used, including global variables, local variables, parameters, etc. It's just that, except for a field/element, the addition of BITS... will have no effect, in other words, it won't change that number of bits or alignment the compiler actually allocates, from what the compiler would do if the type were given as just T. It will not make the declaration illegal. My recommendation not to use BITS in a named TYPE declaration was only because: 1) Outside of a record/object/array, it will make no difference anyway, and 2) Inside of a record/object/array, the effect it has can and often will be different every time, so you have to think about each such field/element and separately judge whether the BITS specification is right for that particular case. And that depends on what comes before it in the record/object/array. Also, there is full assignability between BITS n FOR T and T. So you don't have to put explicit conversions in. The one case there is not such assignability is between BITS n FOR T and BITS m FOR T, with n#m. So copying between two record fields of the same base type but different explicit BITS specifications of different sizes would require explicit conversion. > This is not only un-orthogonal but also a very practical problem; Suggesting that we call > procedures with a parameter of type pixel we will never be able to change that type f.i. to > a packed record with members red, green, blue alpha if there are places in our main > program where we need to declare VAR pix1, pix2 : INTEGER just because Pixel is a packed > type being refused as global variable. > >> >> But, with the BITS specification, the compiler is not allowed to insert alignment padding, (if it >> were, you could not fully specify the layout) and if the field comes at a place that is not 32-bit >> aligned, you will either get a compile-time error or maybe a compiler will accept it and handle >> unaligned field access (it is not required to--ours does not), neither of which is probably what >> you want. Without BITS, the compiler will pad as needed. With BITS, you will have to insert explicit >> padding for alignment yourself as needed, which can be different for every field of this type of every >> record/object. And later adding/removing/changing any prior field can upset it and require you to >> rework the padding. Moreover, I am sure there are simple cases where the required padding differs >> between 32-bit and 64-bit machines. > On the long term I would personally even advocate an ALIGN directive: > f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END; > > It would be very handy to have this for tapping the full power of the vectorization units > of current processors. Manually re-aligning heap allocated objects in an unsafe module > is not a very satisfying alternative (no local/global/stack allocation, memory waste and > thus a certain performance loss if there are many wholes). However this would possibly > also make changes to our garbage collector necessary as it currently only guarantees > word alignment. Finally it would also be possible to write ALIGN BITSIZE(TARGETINT) .... > > >> >> Having been bitten by this more than once, I now only put BITS..FOR as the anonymous type of a specific >> field, never in a named type to be used in multiple places. > A less bitchy and more orthogonal Modula-3 compiler supporting integer globals and > locals of non word size and alignment would be a real pleasure, do you consent Rodney? >> >> (Actually, BITS does the same thing when used as the type of an array element, but cases where this >> matters are rare. If the bit count evenly divides the word size, there won't be any alignment padding >> needed, and BITS won't matter. Otherwise, the compiler is likely to refuse, unless the entire array >> fits in a single machine word. I do have a couple of such cases.) >> >>> The value range - bitsize correlation problem could be solved easily by automatically extending and reducing the value >>> range of an integer type to what its binary representation allows as long as no explicit range has ever been specified for >>> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR [-32768..+32767]). >>> >>> Finally it needs to be said that the argument that target size arithmetics is always the fastest which was often used in >>> here can be very wrong. It does not hold for the 64bit systems of today because the CPU is no more the bottleneck. In >>> a fact now the memory hierarchy has become the new bottleneck (which means that using more memory for the same >>> tends to be much slower). >>> >>> Regards, >>> Elmar >>> >> > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 9 03:44:07 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 8 Jun 2015 18:44:07 -0700 Subject: [M3devel] m3cg right shift signed? Message-ID: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> (Not at propercomputer but sending anyway.) Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? - Jay From hendrik at topoi.pooq.com Tue Jun 9 14:38:29 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 9 Jun 2015 08:38:29 -0400 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> Message-ID: <20150609123829.GA7874@topoi.pooq.com> On Mon, Jun 08, 2015 at 06:44:07PM -0700, Jay wrote: > (Not at propercomputer but sending anyway.) > > Is right shift of a signed type meant to be supported and well > defined in the IR? I thought not, and assert so in the C backend. But > it occurs now, from m3front/cg/set_range or such. Maybe change that > to not? Surely unsigned types are adequate here? Signed and unsigned right shifts are different on most hardware that supports them. Signed shifts do sign extension, so that the shifts of negative numbers remain negative. Unsigned shifts do zero extension. There are (once were?) machines where shifts are (were?) faster than division by powers of two. Not sure how this relates to whatever in Modula 3 might or might not require them, though. -- hendrik From jay.krell at cornell.edu Tue Jun 9 17:42:59 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 15:42:59 +0000 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <20150609123829.GA7874@topoi.pooq.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com>, <20150609123829.GA7874@topoi.pooq.com> Message-ID: Division is among the slowest instructions that operates on integers even on modern hardware. It is/was frequently completely absent, though maybe those days are past. 6502 and 65816 had no hardware multiply or divide, nor maybe signed shift. - Jay > Date: Tue, 9 Jun 2015 08:38:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3cg right shift signed? > > On Mon, Jun 08, 2015 at 06:44:07PM -0700, Jay wrote: > > (Not at propercomputer but sending anyway.) > > > > Is right shift of a signed type meant to be supported and well > > defined in the IR? I thought not, and assert so in the C backend. But > > it occurs now, from m3front/cg/set_range or such. Maybe change that > > to not? Surely unsigned types are adequate here? > > Signed and unsigned right shifts are different on most hardware that > supports them. Signed shifts do sign extension, so that the shifts of > negative numbers remain negative. Unsigned shifts do zero extension. > > There are (once were?) machines where shifts are (were?) faster > than division by powers of two. > > Not sure how this relates to whatever in Modula 3 might or might not > require them, though. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 9 18:05:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 09 Jun 2015 11:05:51 -0500 Subject: [M3devel] m3cg right shift signed? In-Reply-To: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com> Message-ID: <55770EDF.4030606@lcwb.coop> I'm a little unclear what you are asking. I see several places in M3CG_Ops.i3 which call for sign-extension, e.g., load, widen, extract* and any way of pushing on the IR stack. While signed right shift is clearly a good primitive to sign-extend, all these IR ops leave leave this to the code generator, which is good, because the available machine opcodes will be target dependent. As for set_range, this too leaves it to the code generator when the word whose bits are to be set is 32 or 64 bits. For smaller, CG lowers this itself, using more primitive bit-twiddling operators. But it looks like the shifts it generates must be zero-filling, to work right. As for the shift IR operators themselves, M3CG_Ops.i3 defines them as the same as the shifts in Word, none of which is sign-filling. On 06/08/2015 08:44 PM, Jay wrote: > (Not at propercomputer but sending anyway.) > > Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? > > - Jay > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 9 20:30:17 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 18:30:17 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? Message-ID: Is there any interest in maintaining 1) gcc/m3cg/parse.c support for gcc 4.3/4.5/4.6? It is slightly messy. 1b) gcc 4.2? This is where hypothetical ARM_DARWIN support is, last I checked, years ago. 2) gcc-4.3/4.5/4.6 in-tree? Or remove them to keep checkouts smaller? There was complaint as to our source tree size, and these directories do add a lot of size for little gain. They were historically useful to transition versions. I don't think it was wrong to have the temporary growth. All targets except for ARM_DARWIN default to gcc 4.7. Do people often/ever go back and compare/debug? Or want to retain that ability, with the current ease? Or ok to get the files from history, for that rare-to-never event? If we do maintain the gcc backend, I would likely import as gcc-5.1 etc., and recreate the problem, but it can be a "rolling" problem and fix. But I'm not keen on maintaining the gcc backend anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 20:32:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 18:32:03 +0000 Subject: [M3devel] ARM_DARWIN? Message-ID: ARM_DARWIN has never fully worked.I got to the point of c_compile/assemble/link cm3 on the target and then running cm3 failed.I don't remember why. I no longer have a system/toolset setup to test that. Does anyone have an iPhone or iPad (or simulator?) setup with ssh access and native C compiler, assembler, and linker?If so I'd like access to resume/finish that work. And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider deleting gcc-apple. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Jun 9 20:58:20 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 9 Jun 2015 11:58:20 -0700 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: Message-ID: I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that do the trick? On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: > ARM_DARWIN has never fully worked. > I got to the point of c_compile/assemble/link cm3 on the target and then > running cm3 failed. > I don't remember why. > > > I no longer have a system/toolset setup to test that. > > > Does anyone have an iPhone or iPad (or simulator?) setup with ssh access > and native C compiler, assembler, and linker? > If so I'd like access to resume/finish that work. > > > And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider > deleting gcc-apple. > > > Thank you, > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 21:29:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 19:29:41 +0000 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: , Message-ID: Yes that hardware will likely do -- if it can be setup for ssh and w/ native cc/as/ld.But I'd really rather someone else do the setup work. - Jay Date: Tue, 9 Jun 2015 11:58:20 -0700 Subject: Re: [M3devel] ARM_DARWIN? From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that do the trick? On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: ARM_DARWIN has never fully worked.I got to the point of c_compile/assemble/link cm3 on the target and then running cm3 failed.I don't remember why. I no longer have a system/toolset setup to test that. Does anyone have an iPhone or iPad (or simulator?) setup with ssh access and native C compiler, assembler, and linker?If so I'd like access to resume/finish that work. And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider deleting gcc-apple. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Jun 9 21:51:20 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 9 Jun 2015 12:51:20 -0700 Subject: [M3devel] ARM_DARWIN? In-Reply-To: References: Message-ID: Well, according to this < http://guides.macrumors.com/SSH_into_your_iPod_touch_(Windows) > it's jailbreak and then install OpenSSH, then access over wifi. I can do that easily enough. The bigger issue for remove development would be resetting the device. It's battery powered so it requires a physical access to reset. On Tue, Jun 9, 2015 at 12:29 PM, Jay K wrote: > Yes that hardware will *likely *do -- if it can be setup for ssh and w/ > native cc/as/ld. > But I'd really rather someone else do the setup work. > > - Jay > > > > ------------------------------ > Date: Tue, 9 Jun 2015 11:58:20 -0700 > Subject: Re: [M3devel] ARM_DARWIN? > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > > > I have a 2008 iPod Touch (2nd gen I think) which you can have. Will that > do the trick? > > On Tue, Jun 9, 2015 at 11:32 AM, Jay K wrote: > > ARM_DARWIN has never fully worked. > I got to the point of c_compile/assemble/link cm3 on the target and then > running cm3 failed. > I don't remember why. > > > I no longer have a system/toolset setup to test that. > > > Does anyone have an iPhone or iPad (or simulator?) setup with ssh access > and native C compiler, assembler, and linker? > If so I'd like access to resume/finish that work. > > > And then ideally merge m3cc/gcc-apple into m3cc/gcc-4.7 and consider > deleting gcc-apple. > > > Thank you, > - Jay > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Tue Jun 9 22:02:45 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 09 Jun 2015 22:02:45 +0200 Subject: [M3devel] cm3: what are *.mc files Message-ID: <55774665.8060303@elstel.org> What are *.mc - files? They appear in TARGET - directories; most of them are just called _m3main.mc but some of them have other names. I ask because I am writing a program which should recognize and clear object files. It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. Usually files of a specific type start with a 32bit magic; however the mc files all have different starting sequences. Is there still a straight forward way to recognize an .mc file just by its binary content? From jay.krell at cornell.edu Tue Jun 9 22:12:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 20:12:51 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <55774665.8060303@elstel.org> References: <55774665.8060303@elstel.org> Message-ID: I believe before the m3cc backend existed, these were actually C code. (i.e. C backend used to be the one and only) Then they morphed into the intermediate representation fed to cm3cgwhich is what they are now. See here: https://github.com/modula3/cm3/tree/master/m3-sys/m3middle/src/M3CG_BinWr.m3 and m3-sys/m3cc/gcc/gcc/m3cg/parse.c m3cgcat can dump the binary form into text. _m3main.mc is the intermediate file that holds main.There is a configuration option to use it, vs. _m3main.c.I switched to _m3main.c. This is perhaps more portable if using C++ anywhere andwant to get the constructors/destructors run (though all modern implementationsdon't care really). Try cm3 -keep to see them all. i.e. normally they get deleted. - Jay > Date: Tue, 9 Jun 2015 22:02:45 +0200 > From: estellnb at elstel.org > To: m3devel at elegosoft.com > Subject: [M3devel] cm3: what are *.mc files > > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear > object files. > It does not seem to be sufficient to check for uppercase directories > which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by > its binary content? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 9 22:14:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 9 Jun 2015 20:14:03 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: References: <55774665.8060303@elstel.org>, Message-ID: ps: foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io again, see cm3 -keep, err better yet, cm3 -keep -verbose You can see it running cm3cg and as and rm. - Jay From: jay.krell at cornell.edu To: estellnb at elstel.org; m3devel at elegosoft.com Date: Tue, 9 Jun 2015 20:12:51 +0000 Subject: Re: [M3devel] cm3: what are *.mc files I believe before the m3cc backend existed, these were actually C code. (i.e. C backend used to be the one and only) Then they morphed into the intermediate representation fed to cm3cgwhich is what they are now. See here: https://github.com/modula3/cm3/tree/master/m3-sys/m3middle/src/M3CG_BinWr.m3 and m3-sys/m3cc/gcc/gcc/m3cg/parse.c m3cgcat can dump the binary form into text. _m3main.mc is the intermediate file that holds main.There is a configuration option to use it, vs. _m3main.c.I switched to _m3main.c. This is perhaps more portable if using C++ anywhere andwant to get the constructors/destructors run (though all modern implementationsdon't care really). Try cm3 -keep to see them all. i.e. normally they get deleted. - Jay > Date: Tue, 9 Jun 2015 22:02:45 +0200 > From: estellnb at elstel.org > To: m3devel at elegosoft.com > Subject: [M3devel] cm3: what are *.mc files > > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear > object files. > It does not seem to be sufficient to check for uppercase directories > which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by > its binary content? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 10 02:21:23 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 09 Jun 2015 19:21:23 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <55774665.8060303@elstel.org> References: <55774665.8060303@elstel.org> Message-ID: <55778303.3010009@lcwb.coop> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: > What are *.mc - files? > They appear in TARGET - directories; > most of them are just called _m3main.mc but some of them have other names. > > I ask because I am writing a program which should recognize and clear object files. > It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. > > Usually files of a specific type start with a 32bit magic; > however the mc files all have different starting sequences. > > Is there still a straight forward way to recognize an .mc file just by its binary content? > They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, or 16_FD 16_10 16_01, produced by a very recent head compiler. Ignore the 4th byte. > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 10 11:12:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:12:06 +0000 Subject: [M3devel] AMD64_DARWIN config Message-ID: I'd like to bring this back to what it was: diff --git a/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN b/m3-sys/cminstall/src/config-no-install/AMD64_DARWINindex c30dd35..8fa0a4a 100644--- a/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN+++ b/m3-sys/cminstall/src/config-no-install/AMD64_DARWIN@@ -4,9 +4,13 @@ %readonly SYSTEM_CC = "/opt/local/bin/gcc-mp-4.7" %readonly SYSTEM_LIBTOOL = "/opt/local/bin/libtool"-readonly SYSTEM_CC = "/usr/bin/gcc"-readonly SYSTEM_LIBTOOL = "/usr/bin/libtool"-readonly SYSTEM_ASM = "/usr/bin/as"++% Darwin.common does a good job of determining these, esp. the assembler.+% Older /usr/bin/as default to x86 and fail for amd64.+%readonly SYSTEM_CC = "/usr/bin/gcc"+%readonly SYSTEM_LIBTOOL = "/usr/bin/libtool"+%readonly SYSTEM_ASM = "/usr/bin/as"+%readonly SYSTEM_ASM = "/usr/bin/as -arch x86_64" readonly TARGET = "AMD64_DARWIN" readonly GNU_PLATFORM = "i686-darwin8" % "cpu-os" string for GNU Does that really not work for people? Why?Are the cc/libtool on $PATH not acceptable? /usr/bin/as does not work on older systems. At least not w/o -arch. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 11:45:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:45:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? Message-ID: set_range appears to fail to set any bitsif requested to set one bit in a set that doesn't fit in machine word. Test case is here:https://github.com/modula3/cm3/tree/master/m3-sys/m3tests/src/p2/p258 Can anyone try this on a very old cm3 version, like circa 3.6/4.1/5.1?I believe I rewrote this code a few years ago... :( Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 12:01:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 10:01:54 +0000 Subject: [M3devel] m3cg right shift signed? In-Reply-To: References: <683D98C3-8ACA-4951-95C8-68470FED6EC1@gmail.com>, Message-ID: You are right, it isn't new. But I think nothing in the system was using it until the 32bit wchar support came in.I added test p257 that exercises it..so I can make sure m3cc/nt386 agree, and then update C backend to match. - Jay > Subject: Re: [M3devel] m3cg right shift signed? > From: hosking at purdue.edu > Date: Mon, 8 Jun 2015 22:00:22 -0400 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I don?t think this is a new occurrence. Surely signed right shift makes sense to retain the sign bit? > > > On Jun 8, 2015, at 9:44 PM, Jay wrote: > > > > (Not at propercomputer but sending anyway.) > > > > Is right shift of a signed type meant to be supported and well defined in the IR? I thought not, and assert so in the C backend. But it occurs now, from m3front/cg/set_range or such. Maybe change that to not? Surely unsigned types are adequate here? > > > > - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 12:04:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 10:04:44 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? In-Reply-To: References: Message-ID: The bulk of this is commited now -- 4.3/4.5/4.6 removal.gcc-apple and gmp remain, for now. Any objections and I can add them back. We can also trim the frontends and libraries from 4.7 and apple. Thank you, - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 9 Jun 2015 18:30:17 +0000 Subject: [M3devel] gcc-4.3/4.5/4.6 ? Is there any interest in maintaining 1) gcc/m3cg/parse.c support for gcc 4.3/4.5/4.6? It is slightly messy. 1b) gcc 4.2? This is where hypothetical ARM_DARWIN support is, last I checked, years ago. 2) gcc-4.3/4.5/4.6 in-tree? Or remove them to keep checkouts smaller? There was complaint as to our source tree size, and these directories do add a lot of size for little gain. They were historically useful to transition versions. I don't think it was wrong to have the temporary growth. All targets except for ARM_DARWIN default to gcc 4.7. Do people often/ever go back and compare/debug? Or want to retain that ability, with the current ease? Or ok to get the files from history, for that rare-to-never event? If we do maintain the gcc backend, I would likely import as gcc-5.1 etc., and recreate the problem, but it can be a "rolling" problem and fix. But I'm not keen on maintaining the gcc backend anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 11:59:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 09:59:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? In-Reply-To: References: Message-ID: Nevermind. I fixed it. I regressed it not in the rewrite in 2010 but in a minor cleanup in 2012. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Wed, 10 Jun 2015 09:45:06 +0000 Subject: [M3devel] set_range setting one bit in non-word size set fails? set_range appears to fail to set any bitsif requested to set one bit in a set that doesn't fit in machine word. Test case is here:https://github.com/modula3/cm3/tree/master/m3-sys/m3tests/src/p2/p258 Can anyone try this on a very old cm3 version, like circa 3.6/4.1/5.1?I believe I rewrote this code a few years ago... :( Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 23:51:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 21:51:57 +0000 Subject: [M3devel] rpath woes In-Reply-To: <557138A9.3000605@marino.st> References: <55709183.4060804@marino.st>, , <5570947F.2080600@marino.st>, <5570D114.40104@marino.st>, , <557138A9.3000605@marino.st> Message-ID: It appears FreeBSD system ld gained -z origin support in 4.7. This probably maps back to a particular GNU ld change/version. This issue bugs me. I want something simple and portable. There seems to be no perfect solution. Options include: 1 drop support for old systems 2 relink or chrpath/patchelf upon install (Apple and I think Solaris have similar utilities) 3 better predict the install path at build time I think this is what we do by default actually, just not for building "distributions" (at least make-dist.py) 4 wrapper scripts that set LD_LIBRARY_PATH; seems a popular suggestion I find anything involving wrapper scripts inelegant 5 use libtool and whatever it does, which is probably I'd guess is relink-upon-install 6 use autoconf to detect -z origin and fallback to ? $ORIGIN has the downside of possibly not working with setuid/setgid. yes: http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+10.1-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+10.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+8.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+7.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+6.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+5.0-RELEASE&arch=default&format=html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+4.7-RELEASE&arch=default&format=html no: http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=0&manpath=FreeBSD+4.0-RELEASE&arch=default&format=html - Jay > Date: Fri, 5 Jun 2015 07:50:33 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] rpath woes > > On 6/5/2015 06:16, Jay K wrote: > > The FreeBSD4 users were surprisingly vocal surprisingly recently. > > So I put some work into it. > > I agree it is an old system. > > It's completely unsupported (for 7 years now) and they don't need the > latest M3 right? Old release still work fine, that should be enough > justification to change this on trunk. > > > > > Linux.common and FreeBSD.common look very similar but I agree different > > and if it is working for you, good, I can ignore it? > > I guess the main thing was "-Wl,-z,origin"? Needed to make $ORIGIN work? > > Was this right for FreeBSD 5 or 6 or 7 or 8? > > yes. I am using the latest binutils (thus latest ld) with M3. The base > ld is ancient and hand maintained (a GPLv3 issue) so I don't know, > Actually, the linux way is opposite: It explicitly defines SYSTEM_LD > rather than use flags for SYSTEM_CC. So rather than flags, we are > talking about the approach. > > Really nobody with a supported FreeBSD release needs to build from > scratch because it exists in ports. If anybody absolutely wants to, > just say "emulates what the port is doing". This is actually easier > said than done because in addition to using the latest binutils, there > are a number of inline substitutions that are done, see post-patch > target here, starting line 78: > > https://svnweb.freebsd.org/ports/head/lang/modula3/Makefile?revision=388555&view=markup > > line 85 is where newer gcc and newer as is set for m3. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 10 23:57:53 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 10 Jun 2015 21:57:53 +0000 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: <556F42FF.9000106@marino.st> References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: By "gcc obsolete targets", I meant that latest gcc might not target systems that we can/do.Not that gcc versions are no longer maintained. Maybe not all that interesting. We had surprisingly recent usage on OSF/1 v4.0. I had wanted to get Irix working. And AIXand I had an older system. Anyway, I forgot about this part of it, and we could targetthem with the C backend instead of gcc. Or make current gcc work. The size is better now? - Jay > Date: Wed, 3 Jun 2015 20:10:07 +0200 > From: adacore at marino.st > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] Are all 5 gcc branches used? > > On 6/3/2015 20:04, Jay K wrote: > > If gcc obsoletes targets we want to keep, we could keep the old > > versions. Not super useful given our usage levels. > > It has. gcc-4.7 is a closed branch. Only gcc 4.8 and later are not > obsolete by that definition, so all 5 of these are obsolete. > > I would definitely encourage to prune as many of these as possible. I > could probably challenge OpenBSD as well, e.g. Assuming you saying "4.2" > based on the base compiler, why are you assuming it must be built by > base compiler? > > By definition, the base compiler is only required to build base. There > are much newer and well maintained versions of gcc in openbsd ports > tree. There's no reason a ports compiler couldn't be used (I assume > this is actually common). > > > Try xz instead of gzip, maybe it halves the size? > > I can't influence github's API. It is what it is. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jun 12 07:21:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 12 Jun 2015 05:21:44 +0000 Subject: [M3devel] mklib disabled Message-ID: > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > release compiler. > 3. mklib is only used on windows targets anyway. > So, omit mklib altogether, except for windows targets. It is true it did not compile with the released non-Windows m3core. I fixed that. It is true it is only used when targeting Windows. However I want this the way it was. In particular, because w/ it disabled, I have some propensity to break Windows-specific aspects of scripts/python, that leaving it enabled reveals. It is pretty cheap. Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 12 15:39:33 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 08:39:33 -0500 Subject: [M3devel] mklib disabled In-Reply-To: References: Message-ID: <557AE115.8000009@lcwb.coop> On 06/12/2015 12:21 AM, Jay K wrote: > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > release compiler. > > 3. mklib is only used on windows targets anyway. > > So, omit mklib altogether, except for windows targets. > > > It is true it did not compile with the released non-Windows m3core. > I fixed that. > > > It is true it is only used when targeting Windows. > > > However I want this the way it was. > In particular, because w/ it disabled, I have some propensity > to break Windows-specific aspects of scripts/python, that leaving > it enabled reveals. It is pretty cheap. > > > Ok? Not sure I understand, but it sounds like you want to put the github repo in a state where it can not be built by the 5-8 release? If so, that is not OK. The github master branch should always be buildable by the most recent release. Otherwise, we are just insultingly unwelcoming to anybody who is not an insider. You are an insider. Just change it back the way you want in your working directory after you do a pull. > - Jay > > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Jun 12 16:17:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 12 Jun 2015 14:17:30 +0000 Subject: [M3devel] mklib disabled In-Reply-To: <557AE115.8000009@lcwb.coop> References: , <557AE115.8000009@lcwb.coop> Message-ID: > > It is true it did not compile with the released non-Windows m3core. > > I fixed that. > > Not sure I understand, but it sounds like you want to put the github repo > in a state where it can not be built by the 5-8 release? No. It compiles with 5.8.6 now. I fixed that. https://github.com/modula3/cm3/commit/78f21828ad80422b467e31f18c779cebc0a580eb fix for bootstrapping from previous release -- copy in everything used from m3core/winnt.i3 - Jay > Date: Fri, 12 Jun 2015 08:39:33 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] mklib disabled > > > > On 06/12/2015 12:21 AM, Jay K wrote: > > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > > release compiler. > > > 3. mklib is only used on windows targets anyway. > > > So, omit mklib altogether, except for windows targets. > > > > > > It is true it did not compile with the released non-Windows m3core. > > I fixed that. > > > > > > It is true it is only used when targeting Windows. > > > > > > However I want this the way it was. > > In particular, because w/ it disabled, I have some propensity > > to break Windows-specific aspects of scripts/python, that leaving > > it enabled reveals. It is pretty cheap. > > > > > > Ok? > > Not sure I understand, but it sounds like you want to put the github repo > in a state where it can not be built by the 5-8 release? If so, that is > not OK. The github master branch should always be buildable by the most recent > release. Otherwise, we are just insultingly unwelcoming to anybody who > is not an insider. > > You are an insider. Just change it back the way you want in your working directory > after you do a pull. > > > > - Jay > > > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From estellnb at elstel.org Fri Jun 12 16:51:53 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 12 Jun 2015 16:51:53 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: References: <55774665.8060303@elstel.org>, Message-ID: <557AF209.6080408@elstel.org> Thanks a lot Rodney and Jay; that will certainly help my implementation. So far all *.mc files found on my machine have the following signature: 16_FD,00,01,{00} except a few text - .mc from PM3 which start alltogether with "begin_unit". Rodney, do you believe that I can rely on the 4th byte to be zero as generated by the Modula-3 middle end. - or would anyone be ready to uphold such a guarantee for the future? Anyone here who has applied "od" on an .mc generated by a very recent compiler? - do they start with 16_FD,10,01,?00? Most binary file types would guarantee a header of at least 4 Byte and it should be more straight forward and secure to check 32bit instead of 24bit if possible. Any suggestions? Am 10.06.15 um 02:21 schrieb Rodney M. Bates: > > > On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >> What are *.mc - files? >> They appear in TARGET - directories; >> most of them are just called _m3main.mc but some of them have other >> names. >> >> I ask because I am writing a program which should recognize and clear >> object files. >> It does not seem to be sufficient to check for uppercase directories >> which are located together with an src directory. >> >> Usually files of a specific type start with a 32bit magic; >> however the mc files all have different starting sequences. >> >> Is there still a straight forward way to recognize an .mc file just >> by its binary content? >> > > They will start with either 16_FD 16_00 16_01, produced by older > versions of cm3, > or 16_FD 16_10 16_01, produced by a very > recent head compiler. > Ignore the 4th byte. Am 09.06.15 um 22:14 schrieb Jay K: > ps: > > foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo > foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io > > again, see cm3 -keep, err better yet, cm3 -keep -verbose > You can see it running cm3cg and as and rm. > > > - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Jun 12 19:28:57 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 12:28:57 -0500 Subject: [M3devel] mklib disabled In-Reply-To: References: , <557AE115.8000009@lcwb.coop> Message-ID: <557B16D9.2090909@lcwb.coop> On 06/12/2015 09:17 AM, Jay K wrote: > > > It is true it did not compile with the released non-Windows m3core. > > > I fixed that. > > > > Not sure I understand, but it sounds like you want to put the github repo > > in a state where it can not be built by the 5-8 release? > > > No. It compiles with 5.8.6 now. I fixed that. OK, I have no problem with that. > > https://github.com/modula3/cm3/commit/78f21828ad80422b467e31f18c779cebc0a580eb > fix for bootstrapping from previous release -- copy in everything used from m3core/winnt.i3 > > - Jay > > > > Date: Fri, 12 Jun 2015 08:39:33 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] mklib disabled > > > > > > > > On 06/12/2015 12:21 AM, Jay K wrote: > > > > https://github.com/modula3/cm3/blob/master/m3-sys/mklib/src/m3makefile > > > > https://github.com/modula3/cm3/commit/3389c3e2b8b320cce060c6620dcd77c4667671f1 > > > > 1. m3-sys/mklib will not compile without things from m3-libs/m3core/src/win32, > > > > 2. m3-libs/m3core/src/win32 will not compile on non-windows targets, using the > > > > release compiler. > > > > 3. mklib is only used on windows targets anyway. > > > > So, omit mklib altogether, except for windows targets. > > > > > > > > > It is true it did not compile with the released non-Windows m3core. > > > I fixed that. > > > > > > > > > It is true it is only used when targeting Windows. > > > > > > > > > However I want this the way it was. > > > In particular, because w/ it disabled, I have some propensity > > > to break Windows-specific aspects of scripts/python, that leaving > > > it enabled reveals. It is pretty cheap. > > > > > > > > > Ok? > > > > Not sure I understand, but it sounds like you want to put the github repo > > in a state where it can not be built by the 5-8 release? If so, that is > > not OK. The github master branch should always be buildable by the most recent > > release. Otherwise, we are just insultingly unwelcoming to anybody who > > is not an insider. > > > > You are an insider. Just change it back the way you want in your working directory > > after you do a pull. > > > > > > > - Jay > > > > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 12 20:00:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Jun 2015 13:00:00 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557AF209.6080408@elstel.org> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> Message-ID: <557B1E20.7090102@lcwb.coop> On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: > > Thanks a lot Rodney and Jay; > that will certainly help my implementation. > > So far all *.mc files found on my machine have the > following signature: > 16_FD,00,01,{00} > > except a few text - .mc from PM3 which start > alltogether with "begin_unit". > > Rodney, do you believe that I can rely on the 4th byte > to be zero as generated by the Modula-3 middle end. - > or would anyone be ready to uphold such a guarantee > for the future? > The 4th byte is not really dependable for the future. It never has had a real magic number. The FD,00,01 is a version number on the binary format, so even it is likely to change. The 4th byte zero is a binary opcode for begin_unit, equivalent to the "begin_unit" in the PM3 text version. I think the most reliable long-term way is just to look for file names *.mc and *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an INTERFACE. These can be regenerated from source and will not be needed once a compile is complete, unless you are into vetting/debugging the compiler. So deleting them is quite safe. I suppose we could add a magic number. We already have a front/back end compatibility change between the release and head compilers. I can do this, if there is consensus we should. How would we choose the number? > Anyone here who has applied "od" on an .mc generated > by a very recent compiler? - do they start with > 16_FD,10,01,?00? > > Most binary file types would guarantee a header of at > least 4 Byte and it should be more straight forward and > secure to check 32bit instead of 24bit if possible. > > Any suggestions? > > > Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >> >> >> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>> What are *.mc - files? >>> They appear in TARGET - directories; >>> most of them are just called _m3main.mc but some of them have other names. >>> >>> I ask because I am writing a program which should recognize and clear object files. >>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>> >>> Usually files of a specific type start with a 32bit magic; >>> however the mc files all have different starting sequences. >>> >>> Is there still a straight forward way to recognize an .mc file just by its binary content? >>> >> >> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >> or 16_FD 16_10 16_01, produced by a very recent head compiler. >> Ignore the 4th byte. > > > Am 09.06.15 um 22:14 schrieb Jay K: >> ps: >> >> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >> >> again, see cm3 -keep, err better yet, cm3 -keep -verbose >> You can see it running cm3cg and as and rm. >> >> >> - Jay >> > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Fri Jun 12 20:51:31 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 12 Jun 2015 20:51:31 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557B1E20.7090102@lcwb.coop> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> Message-ID: <557B2A33.4010402@elstel.org> Am 12.06.15 um 20:00 schrieb Rodney M. Bates: > > > On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: >> >> Thanks a lot Rodney and Jay; >> that will certainly help my implementation. >> >> So far all *.mc files found on my machine have the >> following signature: >> 16_FD,00,01,{00} >> >> except a few text - .mc from PM3 which start >> alltogether with "begin_unit". >> >> Rodney, do you believe that I can rely on the 4th byte >> to be zero as generated by the Modula-3 middle end. - >> or would anyone be ready to uphold such a guarantee >> for the future? >> > > The 4th byte is not really dependable for the future. It never has had > a real magic number. The FD,00,01 is a version number on the binary > format, so even it is likely to change. > > The 4th byte zero is a binary opcode for begin_unit, equivalent > to the "begin_unit" in the PM3 text version. Well, the begin_unit is exactly what I check for when an .mc appears to be text. If 00 encodes begin_unit I believe it should be save to check for FD,00,01,00 and FD,10,01,00. How could an .mc file not start with begin_unit? Wouldn`t that be invalid? - or if it still would be valid I believe we didn`t generate such files, yet. - so if for the future it may start with any other command a fixed 4-byte magic which is not already interpreted would be welcome. Basically any random number should suffice as with 1.000.000 already registered file formats the probability for a clash would just be 1/4000. Nonetheless we could double- check against the database of the "file" program. Not all files have a completely random magic; f.i. pyc (compiled python files) have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be a valid date. However if we can choose things from scratch I would speak for a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers and timestamps in the following (*). It would be beneficial to have at least a cm3-middleend version number encoded since not every backend can be combined with any middle/front-end. * with a version dependent 2-byte header portion I will need a vaildly set current system date to determine whether it is a .pyc of a future version of python. > > I think the most reliable long-term way is just to look for file names > *.mc and > *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an > INTERFACE. These can be regenerated from source and will not be > needed once a > compile is complete, unless you are into vetting/debugging the compiler. > So deleting them is quite safe. Not all *.mc belong to Modula-3. I have some *.mc in my home directory which are in a fact MS Visual Studio files. That is why I prefer a combination of the file extension and file header/magic to determine whether a file can be auto- matically deleted. For Modula-3 it is also quite save to look for TARGET directories**. However if we meet a file which does not contain plain human readable text we may always want to determine in some way where the file stems from and what data it may contain. File suffixes can be stripped by accident (f.i. on an iso9660 file system) or intendedly by will. - and perhaps we do not want to look to deep into a binary before determining what it is (f.i. by a file manager). Even the "file"-tool was already reported to have a security vulnerability ... ** that will at best poorly work on a non-Unix system where file names are not case sensitive. > > I suppose we could add a magic number. We already have a front/back end > compatibility change between the release and head compilers. I can do > this, > if there is consensus we should. How would we choose the number? > > >> Anyone here who has applied "od" on an .mc generated >> by a very recent compiler? - do they start with >> 16_FD,10,01,?00? >> >> Most binary file types would guarantee a header of at >> least 4 Byte and it should be more straight forward and >> secure to check 32bit instead of 24bit if possible. >> >> Any suggestions? >> >> >> Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >>> >>> >>> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>>> What are *.mc - files? >>>> They appear in TARGET - directories; >>>> most of them are just called _m3main.mc but some of them have other >>>> names. >>>> >>>> I ask because I am writing a program which should recognize and >>>> clear object files. >>>> It does not seem to be sufficient to check for uppercase >>>> directories which are located together with an src directory. >>>> >>>> Usually files of a specific type start with a 32bit magic; >>>> however the mc files all have different starting sequences. >>>> >>>> Is there still a straight forward way to recognize an .mc file just >>>> by its binary content? >>>> >>> >>> They will start with either 16_FD 16_00 16_01, produced by older >>> versions of cm3, >>> or 16_FD 16_10 16_01, produced by a very >>> recent head compiler. >>> Ignore the 4th byte. >> >> >> Am 09.06.15 um 22:14 schrieb Jay K: >>> ps: >>> >>> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >>> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >>> >>> again, see cm3 -keep, err better yet, cm3 -keep -verbose >>> You can see it running cm3cg and as and rm. >>> >>> >>> - Jay >>> >> > From jay.krell at cornell.edu Fri Jun 12 22:10:32 2015 From: jay.krell at cornell.edu (Jay) Date: Fri, 12 Jun 2015 13:10:32 -0700 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557AF209.6080408@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> Message-ID: <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> No guarantees on any of this imho. Nor the extension. The files are usually temporary. What are the magic bytes for .c? What is the purpose here? We could add 4 ignored bytes or even a guid but it'd be a waste. - Jay On Jun 12, 2015, at 7:51 AM, Elmar Stellnberger wrote: > > Thanks a lot Rodney and Jay; > that will certainly help my implementation. > > So far all *.mc files found on my machine have the > following signature: > 16_FD,00,01,{00} > > except a few text - .mc from PM3 which start > alltogether with "begin_unit". > > Rodney, do you believe that I can rely on the 4th byte > to be zero as generated by the Modula-3 middle end. - > or would anyone be ready to uphold such a guarantee > for the future? > > Anyone here who has applied "od" on an .mc generated > by a very recent compiler? - do they start with > 16_FD,10,01,?00? > > Most binary file types would guarantee a header of at > least 4 Byte and it should be more straight forward and > secure to check 32bit instead of 24bit if possible. > > Any suggestions? > > > Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >> >> >> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>> What are *.mc - files? >>> They appear in TARGET - directories; >>> most of them are just called _m3main.mc but some of them have other names. >>> >>> I ask because I am writing a program which should recognize and clear object files. >>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>> >>> Usually files of a specific type start with a 32bit magic; >>> however the mc files all have different starting sequences. >>> >>> Is there still a straight forward way to recognize an .mc file just by its binary content? >> >> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >> or 16_FD 16_10 16_01, produced by a very recent head compiler. >> Ignore the 4th byte. > > > Am 09.06.15 um 22:14 schrieb Jay K: >> ps: >> >> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >> >> again, see cm3 -keep, err better yet, cm3 -keep -verbose >> You can see it running cm3cg and as and rm. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sat Jun 13 01:07:58 2015 From: adacore at marino.st (John Marino) Date: Sat, 13 Jun 2015 01:07:58 +0200 Subject: [M3devel] Are all 5 gcc branches used? In-Reply-To: References: <556EEBB2.30809@marino.st>, , <556F42FF.9000106@marino.st> Message-ID: <557B664E.4040801@marino.st> On 6/10/2015 23:57, Jay K wrote: > By "gcc obsolete targets", I meant that latest gcc might not target > systems that we can/do. > Not that gcc versions are no longer maintained. > Maybe not all that interesting. > > > We had surprisingly recent usage on OSF/1 v4.0. I had wanted to get Irix > working. And AIX > and I had an older system. > > Anyway, I forgot about this part of it, and we could target > them with the C backend instead of gcc. Or make current gcc work. > > > The size is better now? Yes. It reduced the distribution file from Github from 150Mb to 91Mb, a significant reduction of 59Mb (almost 40%) Thanks, John From estellnb at elstel.org Sat Jun 13 10:40:40 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Sat, 13 Jun 2015 10:40:40 +0200 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> Message-ID: <557BEC88.6020608@elstel.org> Am 12.06.15 um 22:10 schrieb Jay: > No guarantees on any of this imho. Nor the extension. The files are > usually temporary. What are the magic bytes for .c? What is the > purpose here? We could add 4 ignored bytes or even a guid but it'd be > a waste. human readable text files do not need any magic but binaries do (though auto-generated text files by a program will need a header, too). A cm3 and middle end version numbers for a compatibilty check if not also a creation timestamp after the magic would be very beneficial, too. It is unprofessional and a bad habit to emit a binary stream without a header and versioning information. Have you never used a ready compiled cm3cg for compiling a new frontend? - There should be a middle end version number to check whether this is prone to fail (and possibly one trying to force application of cm3cg on a newer intermediate code stream. ). -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jun 13 13:37:43 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Jun 2015 07:37:43 -0400 Subject: [M3devel] On magic numbers In-Reply-To: <557B2A33.4010402@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> Message-ID: <20150613113743.GB23865@topoi.pooq.com> On Fri, Jun 12, 2015 at 08:51:31PM +0200, Elmar Stellnberger wrote: > Basically any random > number should suffice as with 1.000.000 already registered file formats the > probability for a clash would just be 1/4000. Nonetheless we could double- > check against the database of the "file" program. For more collision-freeness for the foreseeable future, I'd suggest a 64-bit random number. Even if there were a collision with someone else's 32-bit number, then next 32 bits would likely resolve the issue. It's not too far-fetched to assume that the number of different file formats will continue increasing exponentially even as our world-wide data storage increases. And maybe it's tie that the hash codes we use for data types also increase in length. I've always considered 32 bits a bit too small for this, especially in the days of *huge* program libraries. Maybe a necessary evil as a concession to antiquated linkers, but it could legitimately be made platform-dependent. For backward copatibility, the compiler could just start checking for the magic number. If it's present, skip it. If it's absent, go on as at present. > Not all files have a completely random magic; f.i. pyc (compiled > python files) > have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be > a valid date. However if we can choose things from scratch I would speak for > a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers > and timestamps in the following (*). > It would be beneficial to have at least a cm3-middleend version number > encoded since not every backend can be combined with any middle/front-end. Of course this should still be appended to the 128 (or however many) bits. -- hendrik From hendrik at topoi.pooq.com Sat Jun 13 13:40:59 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Jun 2015 07:40:59 -0400 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557BEC88.6020608@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> <557BEC88.6020608@elstel.org> Message-ID: <20150613114059.GC23865@topoi.pooq.com> On Sat, Jun 13, 2015 at 10:40:40AM +0200, Elmar Stellnberger wrote: > Am 12.06.15 um 22:10 schrieb Jay: > >No guarantees on any of this imho. Nor the extension. The files > >are usually temporary. What are the magic bytes for .c? What is > >the purpose here? We could add 4 ignored bytes or even a guid but > >it'd be a waste. > human readable text files do not need any magic > but binaries do (though auto-generated text files > by a program will need a header, too). > > A cm3 and middle end version numbers for a > compatibilty check if not also a creation timestamp > after the magic would be very beneficial, too. > > It is unprofessional and a bad habit to emit a binary > stream without a header and versioning information. > Have you never used a ready compiled cm3cg for > compiling a new frontend? - There should be a > middle end version number to check whether this > is prone to fail (and possibly one trying to force > application of cm3cg on a newer intermediate code > stream. ). Not to mention the possibility of having a compatibility mode for an old file format, in case it's not too much work. -- hendrik From rodney_bates at lcwb.coop Sat Jun 13 20:23:20 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:23:20 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557B2A33.4010402@elstel.org> References: <55774665.8060303@elstel.org>, <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> Message-ID: <557C7518.6000700@lcwb.coop> On 06/12/2015 01:51 PM, Elmar Stellnberger wrote: > Am 12.06.15 um 20:00 schrieb Rodney M. Bates: >> >> >> On 06/12/2015 09:51 AM, Elmar Stellnberger wrote: >>> >>> Thanks a lot Rodney and Jay; >>> that will certainly help my implementation. >>> >>> So far all *.mc files found on my machine have the >>> following signature: >>> 16_FD,00,01,{00} >>> >>> except a few text - .mc from PM3 which start >>> alltogether with "begin_unit". >>> >>> Rodney, do you believe that I can rely on the 4th byte >>> to be zero as generated by the Modula-3 middle end. - >>> or would anyone be ready to uphold such a guarantee >>> for the future? >>> I doubt anyone would want to guarantee any of these 4 bytes, as they are actual content information and not a magic number. However, I would say the 1st (FD) and last (00, code for begin_unit) have pretty low probability of changing. The middle two contain a version number, so can be expected to change. The second (counting from one) is least significant, and more likely to change. We already have two possible values here, as I reported. >> >> The 4th byte is not really dependable for the future. It never has had >> a real magic number. The FD,00,01 is a version number on the binary >> format, so even it is likely to change. >> >> The 4th byte zero is a binary opcode for begin_unit, equivalent >> to the "begin_unit" in the PM3 text version. > Well, the begin_unit is exactly what I check for when an .mc appears to be text. > If 00 encodes begin_unit I believe it should be save to check for FD,00,01,00 > and FD,10,01,00. How could an .mc file not start with begin_unit? Wouldn`t > that be invalid? - or if it still would be valid I believe we didn`t generate such > files, yet. > - so if for the future it may start with any other command a fixed 4-byte magic > which is not already interpreted would be welcome. Basically any random > number should suffice as with 1.000.000 already registered file formats the > probability for a clash would just be 1/4000. Nonetheless we could double- > check against the database of the "file" program. > Not all files have a completely random magic; f.i. pyc (compiled python files) > have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be > a valid date. However if we can choose things from scratch I would speak for > a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers > and timestamps in the following (*). > It would be beneficial to have at least a cm3-middleend version number > encoded since not every backend can be combined with any middle/front-end. > > * with a version dependent 2-byte header portion I will need a vaildly set current > system date to determine whether it is a .pyc of a future version of python. > >> >> I think the most reliable long-term way is just to look for file names *.mc and >> *.ic. Be sure to look for both. *.mc is for a MODULE and *.ic is for an >> INTERFACE. These can be regenerated from source and will not be needed once a >> compile is complete, unless you are into vetting/debugging the compiler. >> So deleting them is quite safe. > Not all *.mc belong to Modula-3. I have some *.mc in my home directory which > are in a fact MS Visual Studio files. That is why I prefer a combination of the > file extension and file header/magic to determine whether a file can be auto- > matically deleted. OK, file names are not adequate. > For Modula-3 it is also quite save to look for TARGET directories**. However if we > meet a file which does not contain plain human readable text we may always > want to determine in some way where the file stems from and what data it may > contain. File suffixes can be stripped by accident (f.i. on an iso9660 file system) > or intendedly by will. - and perhaps we do not want to look to deep into a > binary before determining what it is (f.i. by a file manager). Even the "file"-tool > was already reported to have a security vulnerability ... > > ** that will at best poorly work on a non-Unix system where file names are not > case sensitive. > >> >> I suppose we could add a magic number. We already have a front/back end >> compatibility change between the release and head compilers. I can do this, >> if there is consensus we should. How would we choose the number? >> >> >>> Anyone here who has applied "od" on an .mc generated >>> by a very recent compiler? - do they start with >>> 16_FD,10,01,?00? >>> >>> Most binary file types would guarantee a header of at >>> least 4 Byte and it should be more straight forward and >>> secure to check 32bit instead of 24bit if possible. >>> >>> Any suggestions? >>> >>> >>> Am 10.06.15 um 02:21 schrieb Rodney M. Bates: >>>> >>>> >>>> On 06/09/2015 03:02 PM, Elmar Stellnberger wrote: >>>>> What are *.mc - files? >>>>> They appear in TARGET - directories; >>>>> most of them are just called _m3main.mc but some of them have other names. >>>>> >>>>> I ask because I am writing a program which should recognize and clear object files. >>>>> It does not seem to be sufficient to check for uppercase directories which are located together with an src directory. >>>>> >>>>> Usually files of a specific type start with a 32bit magic; >>>>> however the mc files all have different starting sequences. >>>>> >>>>> Is there still a straight forward way to recognize an .mc file just by its binary content? >>>>> >>>> >>>> They will start with either 16_FD 16_00 16_01, produced by older versions of cm3, >>>> or 16_FD 16_10 16_01, produced by a very recent head compiler. >>>> Ignore the 4th byte. >>> >>> >>> Am 09.06.15 um 22:14 schrieb Jay K: >>>> ps: >>>> >>>> foo.m3 => foo.mc => cm3cg => foo.ms => as => foo.mo >>>> foo.i3 => foo.ic => cm3cg => foo.is => as => foo.io >>>> >>>> again, see cm3 -keep, err better yet, cm3 -keep -verbose >>>> You can see it running cm3cg and as and rm. >>>> >>>> >>>> - Jay >>>> >>> >> > > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Jun 13 20:27:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:27:00 -0500 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <557BEC88.6020608@elstel.org> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com> <557BEC88.6020608@elstel.org> Message-ID: <557C75F4.4000206@lcwb.coop> On 06/13/2015 03:40 AM, Elmar Stellnberger wrote: > Am 12.06.15 um 22:10 schrieb Jay: >> No guarantees on any of this imho. Nor the extension. The files are usually temporary. What are the magic bytes for .c? What is the purpose here? We could add 4 ignored bytes or even a guid but it'd be a waste. > human readable text files do not need any magic > but binaries do (though auto-generated text files > by a program will need a header, too). > > A cm3 and middle end version numbers for a > compatibilty check if not also a creation timestamp > after the magic would be very beneficial, too. > > It is unprofessional and a bad habit to emit a binary > stream without a header and versioning information. > Have you never used a ready compiled cm3cg for > compiling a new frontend? - There should be a > middle end version number to check whether this > is prone to fail (and possibly one trying to force > application of cm3cg on a newer intermediate code > stream. ). Actually, we have had such a version number, from very early. But it had never changed, until recently, when I changed it to detect this very problem. Now, if you use mismatched cm3cg, you will get an informative error message, instead of something similar to "unrecognized op code" (I paraphrased that.) > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Jun 13 20:42:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Jun 2015 13:42:26 -0500 Subject: [M3devel] On magic numbers In-Reply-To: <20150613113743.GB23865@topoi.pooq.com> References: <55774665.8060303@elstel.org> <557AF209.6080408@elstel.org> <557B1E20.7090102@lcwb.coop> <557B2A33.4010402@elstel.org> <20150613113743.GB23865@topoi.pooq.com> Message-ID: <557C7992.4040103@lcwb.coop> On 06/13/2015 06:37 AM, Hendrik Boom wrote: > On Fri, Jun 12, 2015 at 08:51:31PM +0200, Elmar Stellnberger wrote: > >> Basically any random >> number should suffice as with 1.000.000 already registered file formats the >> probability for a clash would just be 1/4000. Nonetheless we could double- >> check against the database of the "file" program. > > For more collision-freeness for the foreseeable future, I'd suggest a > 64-bit random number. Even if there were a collision with someone > else's 32-bit number, then next 32 bits would likely resolve the issue. > There already is a 64-bit "signature" hash of type structure computed in the compiler, but it is only used in pickles. Elsewhere, a 32-bit "UID" is used. It is just the XOR of the halves of the signature. It was very difficult for me to ferret this conclusion out of the code. The UID is also called something else (I don't remember what) in m3linker and the messages it emits, and may even have more names, making for confusing error messages and difficulty understanding the build system. The fingerprint algorithm has comments suggesting a careful design by someone familiar with high-quality hashes. (That's not me!) Using the full 64 bits everywhere would probably create some annoying transitional compatibility problems. > It's not too far-fetched to assume that the number of different file > formats will continue increasing exponentially even as our world-wide > data storage increases. > > And maybe it's tie that the hash codes we use for data types also > increase in length. I've always considered 32 bits a bit too small for > this, especially in the days of *huge* program libraries. Maybe a > necessary evil as a concession to antiquated linkers, but it could > legitimately be made platform-dependent. > > For backward copatibility, the compiler could just start checking for > the magic number. If it's present, skip it. If it's absent, go on as > at present. > >> Not all files have a completely random magic; f.i. pyc (compiled >> python files) >> have xx\r\ndddd as a header where xx is a 2-byte number and dddd must be >> a valid date. However if we can choose things from scratch I would speak for >> a fixed header f.i. FD,10,01,XX and add things like gcc, cm3 version numbers >> and timestamps in the following (*). >> It would be beneficial to have at least a cm3-middleend version number >> encoded since not every backend can be combined with any middle/front-end. > > Of course this should still be appended to the 128 (or however many) > bits. > > -- hendrik > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Jun 14 03:45:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 14 Jun 2015 01:45:19 +0000 Subject: [M3devel] cm3: what are *.mc files In-Reply-To: <20150613114059.GC23865@topoi.pooq.com> References: <55774665.8060303@elstel.org>, , , <557AF209.6080408@elstel.org>, <77B51204-3A95-4747-8DFE-053B2D8FEC27@gmail.com>, <557BEC88.6020608@elstel.org>, <20150613114059.GC23865@topoi.pooq.com> Message-ID: You should never mix versions. There is no need for a compatibilty mode. You get compatibility mode by building/using an older cm3 and cm3cg together. The system is meant to be tightly coupled. The file format exists I believe primarily as a licensing artifact. When you use the C backend, or the NT386 backend, or presumably LLVM backend, you don't get the files. An approximation of them is used in-memory for the C backend and the NT386 backend never materializes the data, they are just temporary call frames, one at a time. Unless you are in a development mode. Then m3cgcat is a useful dumper of them, and it can translate to C calling into the C backend.. The files are not meant to live long. And even if they did -- hypothetical cm3cginterp, cm3cjit, cm3cgrun -- it'd still be tightly coupled. We are free to change enum values and add/remove fields to each enum. The system has changed through time and there is no need to lock these aspects down. What we need to be compatible with it .m3 and .i3 files. They are currency. There is perhaps some need to be compatible with .o/.mo/.io and even .m3.c/.i3.c. The fact that the C backend has a different ABI than the gcc backend, in particular because of nested functions taking the frame pointer as the last C parameter and not in a special register, worries me, and makes me err toward appending "c" to BUILD_DIR and such, and maybe even TARGET. or "csj" for setjmp -- so that later we have "cppeh" C++ with C++ exception handling.. But anyway, imho .mc files are internal, assuming Tony agrees.. - Jay > Date: Sat, 13 Jun 2015 07:40:59 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3: what are *.mc files > > On Sat, Jun 13, 2015 at 10:40:40AM +0200, Elmar Stellnberger wrote: > > (and possibly one trying to force > > application of cm3cg on a newer intermediate code > > stream. ). > > Not to mention the possibility of having a compatibility mode for an > old file format, in case it's not too much work. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 16 10:43:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 16 Jun 2015 08:43:10 +0000 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu> References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu> Message-ID: I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. From: hosking at cs.purdue.edu Date: Thu, 5 Sep 2013 12:05:07 -0400 To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] TextLiteral.MaxBytes again Here is a modest proposal that should resolve this issue. Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). Updated TextLiteral.i3 and TextLiteral.m3 attached. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 16 10:43:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 16 Jun 2015 08:43:42 +0000 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu>, Message-ID: er..the proposal was old, but nevertheless.. From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: RE: [M3devel] TextLiteral.MaxBytes again Date: Tue, 16 Jun 2015 08:43:10 +0000 I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. From: hosking at cs.purdue.edu Date: Thu, 5 Sep 2013 12:05:07 -0400 To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] TextLiteral.MaxBytes again Here is a modest proposal that should resolve this issue. Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). Updated TextLiteral.i3 and TextLiteral.m3 attached. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 17 18:18:15 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 17 Jun 2015 11:18:15 -0500 Subject: [M3devel] TextLiteral.MaxBytes again In-Reply-To: References: , <52289869.4000301@lcwb.coop>, <35F457AE-9FD1-4E16-906E-F3CFAB657102@cs.purdue.edu>, Message-ID: <55819DC7.4090601@lcwb.coop> On 06/16/2015 03:43 AM, Jay K wrote: > er..the proposal was old, but nevertheless.. > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] TextLiteral.MaxBytes again > Date: Tue, 16 Jun 2015 08:43:10 +0000 > > I must have missed this. I commited the "-15" solution. But this proposal sounds better. I need to revisit.. Thank you. > > If you do this, there are a number of places that should be reviewed for fallout. I think something in runtime has some hard-coded value, and several places there and in pickles use TextLiteral. Plus probably a few tests around. Actually, these should probably be looked at even with the latest change, tho' the latter would not require introduction of unsafe code. > > > From: hosking at cs.purdue.edu > Date: Thu, 5 Sep 2013 12:05:07 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] TextLiteral.MaxBytes again > > Here is a modest proposal that should resolve this issue. > Declare TextLiteral.T to have a minimal buf: ARRAY [0..0] OF Byte; > Then use UNSAFE address arithmetic to access the bytes (TextLiteral.m3 already does the bounds checks explicitly using cnt). > > Updated TextLiteral.i3 and TextLiteral.m3 attached. > > > > Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Sep 5, 2013, at 10:42 AM, "Rodney M. Bates" wrote: > There is another issue here. I want TextLiteral.MaxBytes to get a final size > and not change. Every time it changes, it creates compatibility problems by > orphaning any existing pickle files and any compiled programs that write them. > It changes the fingerprint, so a recently-compiled reading program can't find the > type in its own list of known types. This also affects network objects when two > communicating programs are compiled with different TextLiteral versions. > > MaxBytes' particular contribution to the type doesn't otherwise matter, because it's > artificial, but it still undermines reading pickles. > > I have been loading up the pickle reading code with hard-coded historical fingerprints > of various older versions. But it's a pain to keep doing, > tedious to ferret out the > values, and it still only helps those who constantly download and compile the latest > CM3. > > At some point, I am thinking of choosing some likely historical TextLiteral.T fingerprint > and hard-coding the compiler to artificially deliver it , rather than the one computed from > the version-of-the day of TextLiteral. But this too would not help those who would > rather not constantly download the latest, rebuild the compiler, then recompile their > applications. > > On 09/03/2013 01:02 AM, Jay K wrote: >> ok, another question: >> >> >> CONST >> (* DIV BITSIZE should not be here! *) >> (* MaxBytes = LAST (INTEGER) DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); *) >> MaxBytes = 16_7FFFFFFF DIV BITSIZE (Byte) - 7 - 8 * ORD(BITSIZE(INTEGER) = 64); >> >> TYPE >> T = RTHooks.TextLiteral; >> REVEAL >> T = TEXT BRANDED "TextLiteral.T" OBJECT >> cnt : INTEGER; >> buf : ARRAY [0..MaxBytes - 1] OF Byte; >> OVERRIDES ... >> >> >> T is a reference > type. >> Is there a way to state this with no limit? >> >> >> Isn't it already unsafe? >> You know -- the array is usually smaller and the compiler is only going to check against this large size. >> >> >> - Jay > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 19 03:28:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 18 Jun 2015 20:28:43 -0500 Subject: [M3devel] Anybody using Rd.UnGetCharMulti? Message-ID: <5583704B.1040006@lcwb.coop> Does anybody have code using Rd.UnGetCharMulti? I want to change it a bit to unget several characters in a single call. Some existing calls would have to be changed slightly. -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Mon Jun 22 04:44:20 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 21 Jun 2015 19:44:20 -0700 Subject: [M3devel] ARM_LINUX Bootstrap Compiler Message-ID: Does anyone have a working ARM_LINUX compiler they'd care to post? From the list it does seem some people had it running. If so and you need somewhere to upload it https://github.com/modula3/cm3/releases is probably a good place. - Darko -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Jun 22 10:21:42 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 22 Jun 2015 01:21:42 -0700 Subject: [M3devel] ARM_LINUX Bootstrap Compiler In-Reply-To: References: Message-ID: <20150622082142.E7A8A1A2066@async.async.caltech.edu> I do have it for Raspberry Pi and BeagleBone Black. Works great. But I'll have to look for it. Mika Darko Volaric writes: >--047d7b4140882fa505051912409b >Content-Type: text/plain; charset=UTF-8 > >Does anyone have a working ARM_LINUX compiler they'd care to post? From the >list it does seem some people had it running. > >If so and you need somewhere to upload it >https://github.com/modula3/cm3/releases is probably a good place. > >- Darko > >--047d7b4140882fa505051912409b >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Does anyone have a working=C2=A0ARM_LINUX compiler they= >9;d care to post? From the list it does seem some people had it running.v>
If so and you need somewhere to upload it =C2=A0"https://github.com/modula3/cm3/releases">https://github.com/modula3/cm3/re= >leases =C2=A0is probably a good place.

- Darko= >

> >--047d7b4140882fa505051912409b-- From jay.krell at cornell.edu Mon Jun 22 18:07:55 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 22 Jun 2015 16:07:55 +0000 Subject: [M3devel] ARM_LINUX Bootstrap Compiler In-Reply-To: <20150622082142.E7A8A1A2066@async.async.caltech.edu> References: , <20150622082142.E7A8A1A2066@async.async.caltech.edu> Message-ID: Please try the C backend? Like this: Make sure *target* system has working cc/as/ld/make. cd cm3/scripts/python ./boot1.py ARM_LINUX c You will get something like: cm3-boot-ARM_LINUX-d5.10.0-20150622.tar.gz That works for me. The tail will be replaced by time/date. Copy that to the target, extract, cd, make. See if ./cm3 then works. It should print an error, like: -- building in I386_DARWIN --- Fatal Error: duplicate unit: /cm3/pkg/m3core/src/C/Common/CerrnoC.c ../CerrnoC.c and then, like, have Python installed, and: mkdir /usr/local/cm3/bin mv cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH scripts/python/boot2.sh boot2.sh builds everything. If target system doesn't have cc/as/ld/make, then you cantry using cross tools. Alter $PATH or edit the top of the generated Makefile. And then you can use make-dist.py to build (again) and package everythingand upload that? Thanks, - Jay > To: lists at darko.org > Date: Mon, 22 Jun 2015 01:21:42 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] ARM_LINUX Bootstrap Compiler > > I do have it for Raspberry Pi and BeagleBone Black. Works great. > > But I'll have to look for it. > > Mika > > Darko Volaric writes: > >--047d7b4140882fa505051912409b > >Content-Type: text/plain; charset=UTF-8 > > > >Does anyone have a working ARM_LINUX compiler they'd care to post? From the > >list it does seem some people had it running. > > > >If so and you need somewhere to upload it > >https://github.com/modula3/cm3/releases is probably a good place. > > > >- Darko > > > >--047d7b4140882fa505051912409b > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Does anyone have a working=C2=A0ARM_LINUX compiler they= > >9;d care to post? From the list it does seem some people had it running. >v>
If so and you need somewhere to upload it =C2=A0 >"https://github.com/modula3/cm3/releases">https://github.com/modula3/cm3/re= > >leases =C2=A0is probably a good place.

- Darko= > >

> > > >--047d7b4140882fa505051912409b-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 23 03:50:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 23 Jun 2015 01:50:21 +0000 Subject: [M3devel] pthread assertion failure exiting formsedit Message-ID: Every time I exit formsedit on MacOS X: (gdb) runStarting program: /cm3/bin/formsedit Reading symbols for shared libraries +++++++++++++++++++++..... done ****** runtime error:*** <*ASSERT*> failed.*** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216*** Program received signal SIGABRT, Aborted.0x940181ae in semaphore_wait_signal_trap ()(gdb) bt#0 0x940181ae in semaphore_wait_signal_trap ()#1 0x9404a1c6 in _pthread_cond_wait ()#2 0x9408f449 in pthread_cond_wait ()#3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459#4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239#5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581#6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593#7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52#8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62#9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408#10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115#11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; LOOP IF alertable AND self.alerted THEN self.alerted := FALSE; <*ASSERT self.waitingOn = c.mutex*> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Jun 23 10:41:17 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 23 Jun 2015 01:41:17 -0700 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: References: Message-ID: <20150623084117.468E41A2069@async.async.caltech.edu> Has anyone run the thread tester with the arguments I specified a few weeks ago on this OS/compiler version? Jay K writes: >--_cbc4bf8e-1039-413b-b25f-4e44135e099e_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Every time I exit formsedit on MacOS X: > >(gdb) runStarting program: /cm3/bin/formsedit Reading symbols for shared li= >braries +++++++++++++++++++++..... done > >****** runtime error:*** <*ASSERT*> failed.*** file "../src/thread/PT= >HREAD/ThreadPThread.m3"=2C line 216*** From rodney_bates at lcwb.coop Fri Jun 26 18:03:53 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 26 Jun 2015 11:03:53 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: References: Message-ID: <558D77E9.9040403@lcwb.coop> This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed on this target. I am rerunning it, but it has gone all night and is still going. self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to top of stack--confusing, to me) looks irrelevant. *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 *** Program received signal SIGABRT, Aborted. [Switching to Thread 140370469975808 (LWP 17456)] 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) up #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) up #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) Current language: auto; currently c (m3gdb) up #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 20 Cstdlib.abort (); Current language: auto; currently Modula-3 (m3gdb) bt #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 #7 0x00007faaa4bad577 in Crash (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) at ../src/runtime/common/RTException.m3:79 #8 0x00007faaa4bad26f in DefaultBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 #9 0x00007faaa4bad1bf in InvokeBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 #10 0x00007faaa4bbad0a in Raise (act= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 #11 0x00007faaa4bad31c in DefaultBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 #12 0x00007faaa4bad1bf in InvokeBackstop (a= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 #13 0x00007faaa4bbad0a in Raise (act= RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) at ../src/thread/PTHREAD/ThreadPThread.m3:216 #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 (m3gdb) frame 16 #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) at ../src/thread/PTHREAD/ThreadPThread.m3:216 216 <*ASSERT self.waitingOn = c.mutex*> (m3gdb) info arg self = 16_0000000001856130 m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END alertable = TRUE (m3gdb) info loc prev = 16_00007faa8bffeb90 next = 16_00007faaa4bc1db6 (m3gdb) p self^ $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; END; END; END (m3gdb) p c.name $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 (m3gdb) p self.waitingOn $15 = NIL (m3gdb) p c.mutex $16 = 16_00007faa84000e20 (m3gdb) On a previous run, after more poking around in the debugger, I had: (m3gdb) p c.name $12 = (*16_0000000000615b90*) "all editors closed" on 06/22/2015 08:50 PM, Jay K wrote: > Every time I exit formsedit on MacOS X: > > > (gdb) run > Starting program: /cm3/bin/formsedit > Reading symbols for shared libraries +++++++++++++++++++++..... done > > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 > *** > > > Program received signal SIGABRT, Aborted. > 0x940181ae in semaphore_wait_signal_trap () > (gdb) bt > #0 0x940181ae in semaphore_wait_signal_trap () > #1 0x9404a1c6 in _pthread_cond_wait () > #2 0x9408f449 in pthread_cond_wait () > #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 > #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 > #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 > #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 > #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 > #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 > #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 > #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 > #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 > > > WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; > LOOP > IF alertable AND self.alerted THEN > self.alerted := FALSE; > <*ASSERT self.waitingOn = c.mutex*> > > NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. > > > - Jay > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 04:20:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 27 Jun 2015 21:20:34 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: <558D77E9.9040403@lcwb.coop> References: <558D77E9.9040403@lcwb.coop> Message-ID: <558F59F2.8060406@lcwb.coop> Tony, please review/comment on my analysis, as this kind if code is very delicate, and this is the first time I have looked at it. This is in ThreadPThread.m3. It is possible that thread th, waiting on an M3 Condition variable in XWait, line 239, by actually waiting on pthreads condition variable self.cond, can be both Signaled and Alerted, before it reacquires self.mutex. Either of Signal or Alert will pthread_cond_signal self.cond, but the other can acquire self.mutex before th reacquires it, after the first pthread_cond_signal. When th finally gets self.mutex, both self.alerted and self.waitingOn=NIL. I propose to wrap lines 216 thru' 227 inside IF self.waitingOn # NIL THEN ... Presumably, if both Signal and Alert have happened, we want to raise Alerted, rather than just wake up th. On 06/26/2015 11:03 AM, Rodney M. Bates wrote: > This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed > on this target. I am rerunning it, but it has gone all night and is still going. > > self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be > diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to > top of stack--confusing, to me) looks irrelevant. > > > > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 > *** > > > Program received signal SIGABRT, Aborted. > [Switching to Thread 140370469975808 (LWP 17456)] > 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) up > #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) up > #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 > 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) > Current language: auto; currently c > (m3gdb) up > #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 > 20 Cstdlib.abort (); > Current language: auto; currently Modula-3 > (m3gdb) bt > #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 > #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 > #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 > #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 > #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, > msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 > #7 0x00007faaa4bad577 in Crash (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) > at ../src/runtime/common/RTException.m3:79 > #8 0x00007faaa4bad26f in DefaultBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 > #9 0x00007faaa4bad1bf in InvokeBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 > #10 0x00007faaa4bbad0a in Raise (act= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 > #11 0x00007faaa4bad31c in DefaultBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 > #12 0x00007faaa4bad1bf in InvokeBackstop (a= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 > #13 0x00007faaa4bbad0a in Raise (act= > RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 > #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 > #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) > at ../src/thread/PTHREAD/ThreadPThread.m3:216 > #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 > #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 > #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 > #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 > #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 > #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 > (m3gdb) frame 16 > #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) > at ../src/thread/PTHREAD/ThreadPThread.m3:216 > 216 <*ASSERT self.waitingOn = c.mutex*> > (m3gdb) info arg > self = 16_0000000001856130 > m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END > c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END > alertable = TRUE > (m3gdb) info loc > prev = 16_00007faa8bffeb90 > next = 16_00007faaa4bc1db6 > (m3gdb) p self^ > $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; > nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; > stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, > Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; > pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; > END; END; END > (m3gdb) p c.name > $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 > (m3gdb) p self.waitingOn > $15 = NIL > (m3gdb) p c.mutex > $16 = 16_00007faa84000e20 > (m3gdb) > > > On a previous run, after more poking around in the debugger, I had: > > (m3gdb) p c.name > $12 = (*16_0000000000615b90*) "all editors closed" > > on 06/22/2015 08:50 PM, Jay K wrote: > > >> Every time I exit formsedit on MacOS X: >> >> >> (gdb) run >> Starting program: /cm3/bin/formsedit >> Reading symbols for shared libraries +++++++++++++++++++++..... done >> >> >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >> *** >> >> >> Program received signal SIGABRT, Aborted. >> 0x940181ae in semaphore_wait_signal_trap () >> (gdb) bt >> #0 0x940181ae in semaphore_wait_signal_trap () >> #1 0x9404a1c6 in _pthread_cond_wait () >> #2 0x9408f449 in pthread_cond_wait () >> #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 >> #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 >> #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 >> #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 >> #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 >> #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 >> #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 >> #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 >> #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 >> >> >> WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; >> LOOP >> IF alertable AND self.alerted THEN >> self.alerted := FALSE; >> <*ASSERT self.waitingOn = c.mutex*> >> >> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. >> >> >> - Jay >> > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 04:29:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 27 Jun 2015 21:29:41 -0500 Subject: [M3devel] pthread assertion failure exiting formsedit In-Reply-To: <558F59F2.8060406@lcwb.coop> References: <558D77E9.9040403@lcwb.coop> <558F59F2.8060406@lcwb.coop> Message-ID: <558F5C15.4050303@lcwb.coop> I tried this locally (including lines 228 & 229 also) and it does make the current symptom go away. Also, the thread test is not showing any worrisome output, after a few minutes. BTW, I take it this test is not intended to terminate on its own? On 06/27/2015 09:20 PM, Rodney M. Bates wrote: > Tony, please review/comment on my analysis, as this kind if code is very delicate, > and this is the first time I have looked at it. > > This is in ThreadPThread.m3. > > It is possible that thread th, waiting on an M3 Condition variable in XWait, > line 239, by actually waiting on pthreads condition variable self.cond, > can be both Signaled and Alerted, before it reacquires self.mutex. Either of > Signal or Alert will pthread_cond_signal self.cond, but the other can acquire > self.mutex before th reacquires it, after the first pthread_cond_signal. > When th finally gets self.mutex, both self.alerted and self.waitingOn=NIL. > > I propose to wrap lines 216 thru' 227 inside IF self.waitingOn # NIL THEN ... > > Presumably, if both Signal and Alert have happened, we want to raise Alerted, > rather than just wake up th. > > On 06/26/2015 11:03 AM, Rodney M. Bates wrote: >> This reproduces for me on AMD64_LINUX. As I recall, mika's thread test had passed >> on this target. I am rerunning it, but it has gone all night and is still going. >> >> self.waitingOn and c.mutex are revealed only in ThreadPThread.m3, so there is some hope this can be >> diagnosed by looking therein. Everything shallower in the backtrace (debugger's "down" = closer to >> top of stack--confusing, to me) looks irrelevant. >> >> >> >> >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >> *** >> >> >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 140370469975808 (LWP 17456)] >> 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) up >> #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) up >> #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 >> 21 M3WRAP_RETURN_VOID(Cstdlib__abort, abort, (void), ()) >> Current language: auto; currently c >> (m3gdb) up >> #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 >> 20 Cstdlib.abort (); >> Current language: auto; currently Modula-3 >> (m3gdb) bt >> #0 0x00007faaa45a9f77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x00007faaa45ad5e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> #2 0x00007faaa4bc3f68 in Cstdlib__abort () at ../src/C/Common/CstdlibC.c:21 >> #3 0x00007faaa4bba364 in Crash () at ../src/runtime/POSIX/RTOS.m3:20 >> #4 0x00007faaa4baf027 in Crash (msg=NIL) at ../src/runtime/common/RTProcess.m3:65 >> #5 0x00007faaa4bad11b in EndError (crash=TRUE) at ../src/runtime/common/RTError.m3:118 >> #6 0x00007faaa4bace2d in MsgS (file=16_00007faaa4e033c7, line=216, msgA=16_00007faaa4dfeb70, msgB=16_00007faaa4df9900, >> msgC=16_00007faaa4dfeb70) at ../src/runtime/common/RTError.m3:40 >> #7 0x00007faaa4bad577 in Crash (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE, rte=16_00007faaa4df9760) >> at ../src/runtime/common/RTException.m3:79 >> #8 0x00007faaa4bad26f in DefaultBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:39 >> #9 0x00007faaa4bad1bf in InvokeBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 >> #10 0x00007faaa4bbad0a in Raise (act= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 >> #11 0x00007faaa4bad31c in DefaultBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:47 >> #12 0x00007faaa4bad1bf in InvokeBackstop (a= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END, raises=FALSE) at ../src/runtime/common/RTException.m3:25 >> #13 0x00007faaa4bbad0a in Raise (act= >> RECORD exception = 16_00007faaa4df9760; arg = 16_000000000000000c; module = 16_00007faaa4e19160; line = 216; pc = NIL; info0 = NIL; info1 = NIL; un_except = 16_00007faaa4df9760; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:85 >> #14 0x00007faaa4b9782b in ReportFault (module=16_00007faaa4e19160, info=6912) at ../src/runtime/common/RTHooks.m3:111 >> #15 0x00007faaa4bc17e1 in _m3_fault (arg=6912) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) >> at ../src/thread/PTHREAD/ThreadPThread.m3:216 >> #17 0x00007faaa4bbcc4d in AlertWait (m=16_0000000001906318, c=16_0000000001906340) at ../src/thread/PTHREAD/ThreadPThread.m3:247 >> #18 0x000000000040656b in EditorRootApply (root=16_00000000015d93e8) at ../src/FormsEditVBT.m3:272 >> #19 0x00007faaa4bbe71e in RunThread (me=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:518 >> #20 0x00007faaa4bbe3d2 in ThreadBase (param=16_0000000001856130) at ../src/thread/PTHREAD/ThreadPThread.m3:491 >> #21 0x00007faaa4942f6e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 >> #22 0x00007faaa466d9cd in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> (m3gdb) frame 16 >> #16 0x00007faaa4bbc9c7 in XWait (self=16_0000000001856130, m=16_0000000001906318, c=16_0000000001906340, alertable=TRUE) >> at ../src/thread/PTHREAD/ThreadPThread.m3:216 >> 216 <*ASSERT self.waitingOn = c.mutex*> >> (m3gdb) info arg >> self = 16_0000000001856130 >> m = (*16_0000000001906318*) OBJECT mutex = 16_0000000001854cb0; holder = NIL; waiters = NIL; END >> c = (*16_0000000001906340*) OBJECT mutex = 16_00007faa84000e20; waiters = NIL; name = 16_0000000000615b90; END >> alertable = TRUE >> (m3gdb) info loc >> prev = 16_00007faa8bffeb90 >> next = 16_00007faaa4bc1db6 >> (m3gdb) p self^ >> $13 = RECORD frame = 16_00007faa8bffed00; mutex = 16_0000000001854dd0; cond = 16_0000000001854ce0; alerted = FALSE; waitingOn = NIL; >> nextWaiter = NIL; next = 16_00007faa8c00e530; prev = 16_000000000183b9f0; handle = 16_00007faa8bfff700; >> stackbase = 16_00007faa8bffee68; context = NIL; state = Started; slot = 6; floatState = RECORD behavior = {Trap, Trap, Trap, Trap, >> Trap, Trap, Trap}; sticky = {FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; END; heapState = RECORD inCritical = 0; >> pool = RECORD note = Allocated; pure = FALSE; page = 16_0000000001760000; next = 16_00000000017600c8; limit = 16_0000000001770000; >> END; END; END >> (m3gdb) p c.name >> $14 = (*16_0000000000615b90*) Cannot access memory at address 0x615b90 >> (m3gdb) p self.waitingOn >> $15 = NIL >> (m3gdb) p c.mutex >> $16 = 16_00007faa84000e20 >> (m3gdb) >> >> >> On a previous run, after more poking around in the debugger, I had: >> >> (m3gdb) p c.name >> $12 = (*16_0000000000615b90*) "all editors closed" >> >> on 06/22/2015 08:50 PM, Jay K wrote: >> >> >>> Every time I exit formsedit on MacOS X: >>> >>> >>> (gdb) run >>> Starting program: /cm3/bin/formsedit >>> Reading symbols for shared libraries +++++++++++++++++++++..... done >>> >>> >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 216 >>> *** >>> >>> >>> Program received signal SIGABRT, Aborted. >>> 0x940181ae in semaphore_wait_signal_trap () >>> (gdb) bt >>> #0 0x940181ae in semaphore_wait_signal_trap () >>> #1 0x9404a1c6 in _pthread_cond_wait () >>> #2 0x9408f449 in pthread_cond_wait () >>> #3 0x0093aaee in ThreadPThread__pthread_cond_wait (i=0x1400830, j=0x1400800) at ../src/thread/PTHREAD/ThreadPThreadC.c:459 >>> #4 0x00935ade in ThreadPThread__XWait (self=0x14007a0, m=0x7e0084, c=0x1543644, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:239 >>> #5 0x00937a54 in ThreadPThread__XJoin (self=0x14007a0, t=0x1543628, alertable=0 '\0') at ../src/thread/PTHREAD/ThreadPThread.m3:581 >>> #6 0x00937b5c in Thread__Join (t=0x1543628) at ../src/thread/PTHREAD/ThreadPThread.m3:593 >>> #7 0x00014392 in FormsEdit__main () at ../src/FormsEdit.m3:52 >>> #8 0x0001451d in FormsEdit_M3 (mode=1) at ../src/FormsEdit.m3:62 >>> #9 0x00927175 in RTLinker__RunMainBody (m=0x15c40) at ../src/runtime/common/RTLinker.m3:408 >>> #10 0x00926601 in RTLinker__AddUnitI (m=0x15c40) at ../src/runtime/common/RTLinker.m3:115 >>> #11 0x0092667c in RTLinker__AddUnit (b=0x143a8) at ../src/runtime/common/RTLinker.m3:124 >>> >>> >>> WITH r = pthread_mutex_lock(self.mutex) DO <*ASSERT r=0*> END; >>> LOOP >>> IF alertable AND self.alerted THEN >>> self.alerted := FALSE; >>> <*ASSERT self.waitingOn = c.mutex*> >>> >>> NOTE: The Modula-3 assert does not break into the debugger. There is a later problem in pthread_cond_wait that does. This is with the gcc backend. >>> >>> >>> - Jay >>> >> > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Jun 28 19:58:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 28 Jun 2015 12:58:51 -0500 Subject: [M3devel] pthread scheduling fairness of Signal Message-ID: <559035DB.3040101@lcwb.coop> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really what we want? -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 29 15:40:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 29 Jun 2015 08:40:48 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: References: <559035DB.3040101@lcwb.coop> Message-ID: <55914AE0.8080407@lcwb.coop> On 06/28/2015 01:16 PM, Antony Hosking wrote: > It mirrors the user-level threads information as far as I recall. > Do you think that was intentional in the original user-level implementation? It seems like an open invitation to starvation. And it seems even more important for condition variables than mutexen. (hush, spell checker!) Mika's high-volume thread tester continuously churns out "Thread appears starved or deadlocked" messages by the hundreds. > Sent from my iPad > >> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: >> >> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I >> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those >> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really >> what we want? >> -- >> Rodney Bates >> rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From mika at async.caltech.edu Mon Jun 29 17:31:44 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 29 Jun 2015 08:31:44 -0700 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <55914AE0.8080407@lcwb.coop> References: <559035DB.3040101@lcwb.coop> <55914AE0.8080407@lcwb.coop> Message-ID: <20150629153144.B49851A2062@async.async.caltech.edu> Well code shouldn't depend on fairness. I don't think it's a major issue if it's backwards. I didn't add the warning messages about starved threads to the thread tester. "Rodney M. Bates" writes: > > >On 06/28/2015 01:16 PM, Antony Hosking wrote: >> It mirrors the user-level threads information as far as I recall. >> > >Do you think that was intentional in the original user-level implementation? >It seems like an open invitation to starvation. And it seems even more >important for condition variables than mutexen. (hush, spell checker!) > >Mika's high-volume thread tester continuously churns out >"Thread appears starved or deadlocked" messages by the hundreds. > >> Sent from my iPad >> >>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote >: >>> >>> In vetting ThreadPThread in conjunction with the formsedit assertion failur >e, I >>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but >those >>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this r >eally >>> what we want? >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >> > >-- >Rodney Bates >rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 30 02:18:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 29 Jun 2015 19:18:41 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu> References: <559035DB.3040101@lcwb.coop> <55914AE0.8080407@lcwb.coop> <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu> Message-ID: <5591E061.8090400@lcwb.coop> On 06/29/2015 08:53 AM, Antony Hosking wrote: > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: >> >> >> >> On 06/28/2015 01:16 PM, Antony Hosking wrote: >>> It mirrors the user-level threads information as far as I recall. >>> >> >> Do you think that was intentional in the original user-level implementation? >> It seems like an open invitation to starvation. And it seems even more >> important for condition variables than mutexen. (hush, spell checker!) > > Perhaps take a look at the original user-level implementation to confirm. > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening Mutex waiters and LIFO for Condition waiters. Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. (n=# of waiting threads.) I think n will most often be quite small, but occasionally, maybe not, and if/when high thread counts do occur, it would be a good time for things to be fast. It would be quite easy to make both cases FIFO and O(1). No synchronization changes, just change a few lines of code while holding the same locks that are already being held. If nobody objects, I think I will try this. >> >> Mika's high-volume thread tester continuously churns out >> "Thread appears starved or deadlocked" messages by the hundreds. >> >>> Sent from my iPad >>> >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: >>>> >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really >>>> what we want? >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 30 08:37:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 30 Jun 2015 06:37:50 +0000 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: <5591E061.8090400@lcwb.coop> References: <559035DB.3040101@lcwb.coop>, , <55914AE0.8080407@lcwb.coop>, <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu>, <5591E061.8090400@lcwb.coop> Message-ID: 1) This sounds good. 2) Can you please apply your analysis to the Win32 version also? Tangentially, I've said both of these before: 3) I'd really like to layer the pthread and Win32 versions over a common interface. 4) Possibly make it much thinner, if Modula-3 semantics are close enough to pthread/win32+cv. - Jay > Date: Mon, 29 Jun 2015 19:18:41 -0500 > From: rodney_bates at lcwb.coop > To: hosking at purdue.edu; mika at async.caltech.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] pthread scheduling fairness of Signal > > > > On 06/29/2015 08:53 AM, Antony Hosking wrote: > > > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: > >> > >> > >> > >> On 06/28/2015 01:16 PM, Antony Hosking wrote: > >>> It mirrors the user-level threads information as far as I recall. > >>> > >> > >> Do you think that was intentional in the original user-level implementation? > >> It seems like an open invitation to starvation. And it seems even more > >> important for condition variables than mutexen. (hush, spell checker!) > > > > Perhaps take a look at the original user-level implementation to confirm. > > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > > > > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening > Mutex waiters and LIFO for Condition waiters. > > Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. > (n=# of waiting threads.) I think n will most often be quite small, but occasionally, > maybe not, and if/when high thread counts do occur, it would be a good time for things > to be fast. > > It would be quite easy to make both cases FIFO and O(1). No synchronization > changes, just change a few lines of code while holding the same locks that are already > being held. > > If nobody objects, I think I will try this. > > >> > >> Mika's high-volume thread tester continuously churns out > >> "Thread appears starved or deadlocked" messages by the hundreds. > >> > >>> Sent from my iPad > >>> > >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: > >>>> > >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I > >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those > >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really > >>>> what we want? > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 30 17:04:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 30 Jun 2015 10:04:12 -0500 Subject: [M3devel] pthread scheduling fairness of Signal In-Reply-To: References: <559035DB.3040101@lcwb.coop>, , <55914AE0.8080407@lcwb.coop>, <64E16D1D-83AA-438A-BBB5-F7135649AF3B@purdue.edu>, <5591E061.8090400@lcwb.coop> Message-ID: <5592AFEC.70906@lcwb.coop> On 06/30/2015 01:37 AM, Jay K wrote: > 1) This sounds good. > 2) Can you please apply your analysis to the Win32 version also? > It looks to me like these matters are entirely delegated to windows-supplied library routines EnterCriticalSection, LeaveCriticalSection, SetEvent, and WaitForMultipleObjects. > > Tangentially, I've said both of these before: > > > 3) I'd really like to layer the pthread and Win32 > versions over a common interface. > > > 4) Possibly make it much thinner, if Modula-3 semantics > are close enough to pthread/win32+cv. > > > - Jay > > > > > Date: Mon, 29 Jun 2015 19:18:41 -0500 > > From: rodney_bates at lcwb.coop > > To: hosking at purdue.edu; mika at async.caltech.edu; m3devel at elegosoft.com > > Subject: Re: [M3devel] pthread scheduling fairness of Signal > > > > > > > > On 06/29/2015 08:53 AM, Antony Hosking wrote: > > > > > >> On Jun 29, 2015, at 9:40 AM, Rodney M. Bates wrote: > > >> > > >> > > >> > > >> On 06/28/2015 01:16 PM, Antony Hosking wrote: > > >>> It mirrors the user-level threads information as far as I recall. > > >>> > > >> > > >> Do you think that was intentional in the original user-level implementation? > > >> It seems like an open invitation to starvation. And it seems even more > > >> important for condition variables than mutexen. (hush, spell checker!) > > > > > > Perhaps take a look at the original user-level implementation to confirm. > > > I agree that ideally it would be FIFO. Feel free to think on what is needed to do that. > > > > > > > I looked at ThreadPosix.m3, and indeed it is the same way: FIFO for awakening > > Mutex waiters and LIFO for Condition waiters. > > > > Also, in both thread implementations, the FIFO wakeup is O(n) for a single wakeup. > > (n=# of waiting threads.) I think n will most often be quite small, but occasionally, > > maybe not, and if/when high thread counts do occur, it would be a good time for things > > to be fast. > > > > It would be quite easy to make both cases FIFO and O(1). No synchronization > > changes, just change a few lines of code while holding the same locks that are already > > being held. > > > > If nobody objects, I think I will try this. > > > > >> > > >> Mika's high-volume thread tester continuously churns out > > >> "Thread appears starved or deadlocked" messages by the hundreds. > > >> > > >>> Sent from my iPad > > >>> > > >>>> On Jun 28, 2015, at 1:58 PM, Rodney M. Bates wrote: > > >>>> > > >>>> In vetting ThreadPThread in conjunction with the formsedit assertion failure, I > > >>>> notice that threads waiting on an M3 MUTEX are awakened in FIFO order, but those > > >>>> waiting on an M3 Condition variable are awakened LIFO by Signal. Is this really > > >>>> what we want? > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> > > >> > > >> -- > > >> Rodney Bates > > >> rodney.m.bates at acm.org > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org