From wagner at elegosoft.com Tue May 8 10:18:11 2018 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 8 May 2018 10:18:11 +0200 Subject: [M3devel] Coroutines... --> m3devel up again In-Reply-To: <5119064B-678E-4E69-9CF1-D6CFCE94B591@purdue.edu> References: <6FA04EBB-62AD-4C1A-BB29-24B53223704E@purdue.edu> <5119064B-678E-4E69-9CF1-D6CFCE94B591@purdue.edu> Message-ID: <20180508101811.deda40e4356792f476d78c0c@elegosoft.com> On Mon, 7 May 2018 23:22:13 +0000 "Hosking, Antony L" wrote: > PS I responded to Mika in another thread. > > Sent from my iPad > > On May 8, 2018, at 9:21 AM, Hosking, Antony L > wrote: > > I have a feeling it is down... :-( > > Sent from my iPad > > On May 8, 2018, at 3:40 AM, Mika Nystrom > wrote: > > Hello everyone, is m3devel down? Hope you didn't kill it permanently! > > Mika We seem to have a memory shortage on the current server for m3devel. Migrating the m3 stuff has been on the list for some time and will be done in a few weeks. See attached mail. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An embedded message was scrubbed... From: Navid Afkhami Subject: Modula 3 is up and running again Date: Tue, 8 May 2018 10:09:08 +0200 Size: 1526 URL: From nafkhami at elegosoft.com Tue May 8 11:47:20 2018 From: nafkhami at elegosoft.com (Navid Afkhami) Date: Tue, 8 May 2018 11:47:20 +0200 Subject: [M3devel] Test Mail from elegosoft administration Message-ID: <87e43126-98db-af6d-adec-4d30bc9b131a@elegosoft.com> Test -- Navid Afkhami IT Services & Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 navid.afkhami at elegosoft.com fax +49 30 23 45 86 95 https://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From jayk123 at hotmail.com Sat May 12 10:30:07 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 08:30:07 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: Message-ID: Sorry, I've really been meaning to get back into this. Try https://github.com/modula3/cm3/blob/master/scripts/python/make-dist.py If it works, extract the .tar.gz or .zip somewhere, set PATH to it, and try it out. or make a copy of your install either a backup, or set $PATH/$CM3 to the copy and scripts/python/upgrade.py or scripts/python/upgrade-full.sh Upgrade will overwrite the CM3 and runtime it uses, before it knows if things are working. make-dist does not overwrite anything, but it does build "twice", with itself, so you can have confidence in the result. When I wrote upgrade.py, I misread what I was reimplementing, and made it minimal (compiler/runtime). upgrade-full.sh mimics the other upgrade.sh -- compile/runtime, and then everything. The python scripts handle building and shipping m3cc. They don't make you cd or rm and they know the order, etc. I'd really like to move to C or C++ backend, and maybe write my own LLVM backend someday.. - Jay ________________________________ From: mika.nystrom at gmail.com on behalf of Mika Nystrom Sent: Saturday, May 12, 2018 8:16 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 10:35:35 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 08:35:35 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , Message-ID: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 10:57:46 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 08:57:46 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , Message-ID: Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 10:59:10 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 08:59:10 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , Message-ID: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:03:44 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:03:44 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , , Message-ID: Could be your first attempt broke your install. I need to run this stuf again/soon, sorry I've been away so much. ? - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 8:59 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:09:20 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:09:20 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , Message-ID: I tried to avoid depending on newer Python -- years ago. > AttributeError: 'NoneType' object has no attribute 'endswith' is a familiar error to me. It either means cm3 crashed, most common, or maybe a very old cm3 that doesn't reports its host/target. I should make it more verbose -- rerun cm3 w/o redirection. What does running: /p/hlp/utils/cm3/bin/cm3 show? 2.4 is a bit old sounding...try 2.5 or 6 or 7?? It is true, software is too incompatible. It is hard to keep everything working. Implementation becomes interface.. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:06 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:27:56 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:27:56 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , Message-ID: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:34:21 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:34:21 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , Message-ID: You can fiddle with the script to copy more. This looks like self inflicted wound, though I am curious why the dependencies. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:27 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:40:16 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:40:16 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , Message-ID: I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:41:30 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:41:30 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , Message-ID: Remove cvsup from here: https://github.com/modula3/cm3/blob/master/scripts/pkginfo.txt and commit it. If someone wants it they can fix it and restore it. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:40 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 11:48:13 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 09:48:13 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , , Message-ID: Caused by: https://github.com/modula3/cm3/commit/4aa3a9f9ebab8b048f5d067473500abf65e7a8e1 - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:41 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Remove cvsup from here: https://github.com/modula3/cm3/blob/master/scripts/pkginfo.txt and commit it. If someone wants it they can fix it and restore it. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:40 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 12:15:20 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 10:15:20 +0000 Subject: [M3devel] darwin/stabs/git Message-ID: 1. I'm a git expert now, I guess, kinda. Still just commit to master? Start requiring pull requests? 2. Stabs was never useful on Darwin. No Mac debugger reads them -- m3gdb doesn't run on Mac I beliee. Now it is worse. I thought I had disabled them, on Darwin. I'll fix it. PR or just commit to master? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 12:21:27 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 10:21:27 +0000 Subject: [M3devel] darwin/stabs/git In-Reply-To: References: Message-ID: Eek this is pretty bad out of box experience...releases don't really work.. ? old gcc seems to always output stabs, and Apple assembler fails. I'll try to fix. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 10:15 AM To: m3devel Subject: darwin/stabs/git 1. I'm a git expert now, I guess, kinda. Still just commit to master? Start requiring pull requests? 2. Stabs was never useful on Darwin. No Mac debugger reads them -- m3gdb doesn't run on Mac I beliee. Now it is worse. I thought I had disabled them, on Darwin. I'll fix it. PR or just commit to master? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 12:24:44 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 10:24:44 +0000 Subject: [M3devel] darwin/stabs/git In-Reply-To: References: , Message-ID: ok, I fixed it a while ago..need to make a release (or use someone else's ?) - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, May 12, 2018 10:21 AM To: m3devel Subject: Re: [M3devel] darwin/stabs/git Eek this is pretty bad out of box experience...releases don't really work.. ? old gcc seems to always output stabs, and Apple assembler fails. I'll try to fix. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 10:15 AM To: m3devel Subject: darwin/stabs/git 1. I'm a git expert now, I guess, kinda. Still just commit to master? Start requiring pull requests? 2. Stabs was never useful on Darwin. No Mac debugger reads them -- m3gdb doesn't run on Mac I beliee. Now it is worse. I thought I had disabled them, on Darwin. I'll fix it. PR or just commit to master? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 14:25:27 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 12:25:27 +0000 Subject: [M3devel] darwin/stabs/git In-Reply-To: References: , , Message-ID: ok, 4 little pull requests from me..there might be some controversy. https://github.com/modula3/cm3/pulls Should see if some level of symbols work... I cheated and built out of order -- current cm3cg worked. ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, May 12, 2018 10:24 AM To: m3devel Subject: Re: [M3devel] darwin/stabs/git ok, I fixed it a while ago..need to make a release (or use someone else's ?) - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, May 12, 2018 10:21 AM To: m3devel Subject: Re: [M3devel] darwin/stabs/git Eek this is pretty bad out of box experience...releases don't really work.. ? old gcc seems to always output stabs, and Apple assembler fails. I'll try to fix. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 10:15 AM To: m3devel Subject: darwin/stabs/git 1. I'm a git expert now, I guess, kinda. Still just commit to master? Start requiring pull requests? 2. Stabs was never useful on Darwin. No Mac debugger reads them -- m3gdb doesn't run on Mac I beliee. Now it is worse. I thought I had disabled them, on Darwin. I'll fix it. PR or just commit to master? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 14:42:42 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 12:42:42 +0000 Subject: [M3devel] release formats? Message-ID: .tar.gz? tar.xz? .zip? .msi? .deb? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 14:44:56 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 12:44:56 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , , , Message-ID: I was able to quickly update/release I386_DARWIN tonight. I'll try some others "soon" -- amd64, linux, windows, probably that's it (no solaris or BSD soon..) and I'll try for some C releases... https://github.com/modula3/cm3/pulls https://github.com/jaykrell/cm3/commits/master System is not super speedy despite hardware advanced but still works ok.. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:48 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Caused by: https://github.com/modula3/cm3/commit/4aa3a9f9ebab8b048f5d067473500abf65e7a8e1 - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:41 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Remove cvsup from here: https://github.com/modula3/cm3/blob/master/scripts/pkginfo.txt and commit it. If someone wants it they can fix it and restore it. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:40 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 14:45:49 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 12:45:49 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , , , , Message-ID: I was able to quickly update/release I386_DARWIN tonight. I'll try some others "soon" -- amd64, linux, windows, probably that's it (no solaris or BSD soon..) and I'll try for some C releases... https://github.com/modula3/cm3/pulls https://github.com/jaykrell/cm3/commits/master https://github.com/modula3/cm3/releases System is not super speedy despite hardware advanced but still works ok.. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:48 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Caused by: https://github.com/modula3/cm3/commit/4aa3a9f9ebab8b048f5d067473500abf65e7a8e1 - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:41 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Remove cvsup from here: https://github.com/modula3/cm3/blob/master/scripts/pkginfo.txt and commit it. If someone wants it they can fix it and restore it. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:40 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 15:00:30 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 13:00:30 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , , , , , Message-ID: and ok..maybe obnoxious to hear this, but I crossed from I386_DARWIN to AMD64_DARWIN easily with boot1.py + boot2.py. Very turnkey. You run ./boot1.py Copy the archive, extract it, cd into it, make, mkdir /cm3/bin, mv cm3 /cm3/bin, PATH=/cm3/bin:$PATH, boot2.py If same machine, skip copy/extract and just cd. boot2.py builds everything, so no need for upgrade or upgrade-full. I'll make-dist next. Then on to C backend and Linux and Windows and maybe BSD/Solaris...sparc/opencsw? - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 12:45 PM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I was able to quickly update/release I386_DARWIN tonight. I'll try some others "soon" -- amd64, linux, windows, probably that's it (no solaris or BSD soon..) and I'll try for some C releases... https://github.com/modula3/cm3/pulls https://github.com/jaykrell/cm3/commits/master https://github.com/modula3/cm3/releases System is not super speedy despite hardware advanced but still works ok.. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:48 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Caused by: https://github.com/modula3/cm3/commit/4aa3a9f9ebab8b048f5d067473500abf65e7a8e1 - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:41 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Remove cvsup from here: https://github.com/modula3/cm3/blob/master/scripts/pkginfo.txt and commit it. If someone wants it they can fix it and restore it. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:40 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat May 12 15:09:05 2018 From: jayk123 at hotmail.com (Jay K) Date: Sat, 12 May 2018 13:09:05 +0000 Subject: [M3devel] Time flies but my problems stay the same! (Compiler bootstrapping issues...) In-Reply-To: References: , , , , , , Message-ID: I put an AMD64_DARWIN release up. We could use some CI. C backend holding in there. It can build m3core and only error on libm3 is: nvertPacking_m.o ../src/pickle/ver2/ConvertPacking.m3:1262:560: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value] ...(INT64) */ L_1034_L_1035))))))&((INT64)( /* cast_removed2: (INT64) */ /* cast_removed2: (INT64) */ INT64_(-1)<<((UINT64)( /* cast_rem... ~~~~~~~~~~^ ../src/pickle/ver2/ConvertPacking.m3:1283:560: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value] ...(INT64) */ L_1050_L_1051))))))&((INT64)( /* cast_removed2: (INT64) */ /* cast_removed2: (INT64) */ INT64_(-1)<<((UINT64)( /* cast_rem... ~~~~~~~~~~^ I'll get to it later.. ________________________________ From: Jay K Sent: Saturday, May 12, 2018 12:45 PM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I was able to quickly update/release I386_DARWIN tonight. I'll try some others "soon" -- amd64, linux, windows, probably that's it (no solaris or BSD soon..) and I'll try for some C releases... https://github.com/modula3/cm3/pulls https://github.com/jaykrell/cm3/commits/master https://github.com/modula3/cm3/releases System is not super speedy despite hardware advanced but still works ok.. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:48 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Caused by: https://github.com/modula3/cm3/commit/4aa3a9f9ebab8b048f5d067473500abf65e7a8e1 - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:41 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Remove cvsup from here: https://github.com/modula3/cm3/blob/master/scripts/pkginfo.txt and commit it. If someone wants it they can fix it and restore it. - Jay ________________________________ From: Jay K Sent: Saturday, May 12, 2018 9:40 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) I have to go for a while. Your Python is ok. I'd see if the object autoconverts to boolean, w/o a compare. Maybe just masking real problem though. 1. You can remove cvsup from the package list (pkginfo.txt?) 2. If these are same/similar enough, cvsup can use libm3. Losing portability to old libm3? 3. Or one of them can be renamed. 4. Since it got this far, you probably have plenty, in /tmp, no need to build more. 5. But it might not have shipped to /tmp. 6. We should fix this. 7. Package names should be able to have dots or slashes or colons. Preferably dots. Or just use longer names. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, May 12, 2018 9:35 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) and indeed: sccj004186> pwd /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python sccj004186> find ../.. -name MD5.i3 ../../m3-libs/libm3/src/hash/MD5.i3 ../../m3-tools/cvsup/suplib/src/MD5.i3 On Sat, May 12, 2018 at 2:35 AM, Mika Nystrom > wrote: the .sos are pulled in by various packages. Oh, this is probably my mistake originally. I believe this is the issue: on every system that I use, all the libraries are in different places. I can't go and update every single m3makefile. So I took a shortcut: I just link every binary with the libraries in question, out of the config file. I do this when I install cm3 in a new environment. As a result, the built binaries have extra dependencies, but you never run into problems with this because everything is built and running on that system. I don't know the "right way to do this", to avoid linking binaries with extra stuff. Happy for advice, of course... ---- In any case after some hacking to work around my self-inflicted problem above, I get a LOT further, and the compile dies at this step: == package /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-tools/cvsup/suplib == +++ /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/bin/cm3 -build -DROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides Fatal Error: duplicate unit: /tmp/tmplf15vM/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180512/pkg/libm3/src/hash/MD5.i3 ../src/MD5.i3 *** execution of [, ] failed *** sccj004186> On Sat, May 12, 2018 at 2:27 AM, Jay K > wrote: This is triply wierd. - Why do we depend on so many .sos? expat? curses? zlib? Something messing around with glibc? Does helloworld+pthreads look similar? On Linux (and Windows), further static linking should work, but it is kinda frowned upon, bloating. - What are they doing in cm3? - How are they referenced? By full path??? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 9:17 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Here's a funny one... an issue I have often had with CM3 in general---that is, dependency on a bunch of random shared libraries from elsewhere---, and what looks like a clever solution (but foiled your scripts, Jay). I or someone like me had put a bunch of .so's in cm3/bin ... (the issue is that in this system, the directory "cm3" (parent of cm3/bin) has to be self-contained. If funny shared libraries are required, they need to be somewhere in this tree.) sccj004186> ldd !!:0 ldd /tmp/tmpZyarxY/compiler_with_previous/bin/cm3 linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => not found libreadline.so.6 => not found libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaaafeb000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab207000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab459000) libtinfo.so.5 => not found libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaab6a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab8cc000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaabaea000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaabe66000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac08c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac291000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaac49a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac69d000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaac8b5000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaacae2000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaaccf1000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaacf0d000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaad111000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaad368000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad709000) sccj004186> ldd `which cm3` linux-vdso.so.1 => (0x00002aaaaaaab000) libm.so.6 => /lib64/libm.so.6 (0x00002aaaaaace000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaaad48000) libssl.so.1.0.0 => /p/hlp/utils/cm3/bin/libssl.so.1.0.0 (0x00002aaaaafea000) libreadline.so.6 => /p/hlp/utils/cm3/bin/libreadline.so.6 (0x00002aaaab24a000) libgssrpc.so.4 => /usr/lib64/libgssrpc.so.4 (0x00002aaaab492000) libldap_r-2.4.so.2 => /usr/lib64/libldap_r-2.4.so.2 (0x00002aaaab6ae000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00002aaaab900000) libtinfo.so.5 => /p/hlp/utils/cm3/bin/libtinfo.so.5 (0x00002aaaabb49000) libexpat.so.1 => /lib64/libexpat.so.1 (0x00002aaaabd72000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaabf9c000) libc.so.6 => /lib64/libc.so.6 (0x00002aaaac1ba000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002aaaac536000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaac75c000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002aaaac961000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002aaaacb6a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaacd6d000) libcrypto.so.1.0.0 => /p/hlp/utils/cm3/bin/libcrypto.so.1.0.0 (0x00002aaaacf85000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad37d000) libz.so.1 => /lib64/libz.so.1 (0x00002aaaad581000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002aaaad798000) liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00002aaaad9c5000) libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002aaaadbd4000) libssl.so.0.9.8 => /usr/lib64/libssl.so.0.9.8 (0x00002aaaaddf0000) libcrypto.so.0.9.8 => /usr/lib64/libcrypto.so.0.9.8 (0x00002aaaae047000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) On Sat, May 12, 2018 at 2:11 AM, Mika Nystrom > wrote: I see what the problem was... with this diff I get a lot further, anyhow: (Note I am not a Python expert and don't know whether my edit is idiomatic.) diff --git i/scripts/python/pylib.py w/scripts/python/pylib.py index 9771791..1c38f44 100755 --- i/scripts/python/pylib.py +++ w/scripts/python/pylib.py @@ -697,7 +697,7 @@ if Target.endswith("_MINGW") or Target.endswith("_NT"): TargetOS = "WIN32" HAVE_SERIAL = True -if Host.endswith("_NT") or Host == "NT386": +if Host != None and (Host.endswith("_NT") or Host == "NT386"): Q = "" # q for quote: This is probably about the host, not the target. #----------------------------------------------------------------------------- @@ -2221,7 +2221,7 @@ def SetVisualCPlusPlus2015OrNewer(): #sys.exit(3) def IsCygwinHostTarget(): # confused - return Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX") + return Host != None and (Host.endswith("_CYGWIN") or (Host == "NT386" and GCC_BACKEND and TargetOS == "POSIX")) def IsMinGWHostTarget(): return (Target == "NT386" and GCC_BACKEND and TargetOS == "WIN32") or Target.endswith("_MINGW") On Sat, May 12, 2018 at 2:06 AM, Mika Nystrom > wrote: I was using Python 2.4. That's indeed very old. With Python 3.6.0 I get: sccj004186> ./make-dist.py Traceback (most recent call last): File "./make-dist.py", line 7, in import pylib File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 962 PackageDB = (PackageDB or map(lambda(a): a.replace("\n", "").replace('\\', '/').replace("\r", ""), open(PKGSDB))) ^ SyntaxError: invalid syntax sccj004186> I can try a binary search on python versions :-) (See one of the reasons I love coding in Modula-3... there aren't enough of us screwing around with it to create versioning problems (for regular coding)!) On Sat, May 12, 2018 at 2:04 AM, Mika Nystrom > wrote: Hm, well ok... I am happy to use your scripts! The build order was something Tony gave me that ought to work (build code generator, then build the libraries in a careful order so as not to screw up the dependencies...) Yes /p/hlp/utils/cm3 is a working install. That's the compiler I've used for years. I use it every day. I just refreshed it from a tarball to make sure I didn't screw anything up in the build attempts earlier, and I get the same results. And yes you are right, mnystroe/git/cm3 is the git clone, utils/cm3 is the working install. My Python is maybe too old. On Sat, May 12, 2018 at 1:59 AM, Jay K > wrote: Sorry, my mistake: > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 is your git clone. I meant: > CM3_INSTALL=/p/hlp/utils/cm3 Is that a working install? - Jay ________________________________ From: Jay K > Sent: Saturday, May 12, 2018 8:57 AM To: Mika Nystrom Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Looking more closely at your steps, they definitely look wrong. When there is no interface change, you can just build/ship anything willy nilly in any order. As you show. But when there are interface changes -- more likely with more time -- you must be more precise. Use the scripts. > /nfs/sc/disks/hlp_0015/mnystroe/git/cm3 Is this a known working cm3? Or the broken one leftover from your first attempt? You must have a working cm3. And its m3core and libm3. I believe you can set $CM3, or we will search $PATH, and it must be "shipped", i.e. so we can find lib and pkg next to it. Though we could be more flexible in that matter. (Some day I'd like to confirm that libm3 doesn't atter, only m3core). - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:52 AM To: Jay K Cc: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) Yes I know you do things with scripts. All right: sccj004186> ./make-dist.py CM3_TARGET=AMD64_LINUX export CM3_TARGET CM3_INSTALL=/p/hlp/utils/cm3 export CM3_INSTALL CM3_ROOT=/nfs/sc/disks/hlp_0015/mnystroe/git/cm3 export CM3_ROOT STAGE=/tmp/tmpVPtFec export STAGE ============================================================================= build new compiler with old compiler and old runtime (/p/hlp/utils/cm3 to /tmp/tmpVPtFec/compiler_with_previous) ============================================================================= ... cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/cm3cfg.common cp -Pv /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-sys/cminstall/src/config-no-install/gnuld.common /tmp/tmpVPtFec/compiler_with_previous/bin/config/gnuld.common Traceback (most recent call last): File "./make-dist.py", line 279, in ? Setup(InstallRoot, InstallRoot_CompilerWithPrevious) File "./make-dist.py", line 275, in Setup reload(pylib) or FatalError() File "/nfs/sc/disks/hlp_0015/mnystroe/git/cm3/scripts/python/pylib.py", line 700, in ? if Host.endswith("_NT") or Host == "NT386": AttributeError: 'NoneType' object has no attribute 'endswith' sccj004186> On Sat, May 12, 2018 at 1:35 AM, Jay K > wrote: I have no idea if what you did is correct. I don't do things that way, by far. I encoded the correct steps in scripts, which were based on others before me, and I have run them countless times. It is not worth debugging anything else. You may have mixed up compiler and runtime. For example, the change to alloca the jmpbuf, so porters no longer have to teach the frontend the size of it. That changed it then from a "value" to a pointer, and getting it wrong is fatal. - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, May 12, 2018 8:31 AM To: m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking Cc: peter.mckinna at gmail.com; rodney_bates at lcwb.coop Subject: Re: Time flies but my problems stay the same! (Compiler bootstrapping issues...) hm, more data: (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf626f0 in __longjmp () from /lib64/libc.so.6 #1 0x00002aaaabf626d3 in siglongjmp () from /lib64/libc.so.6 #2 0x00000000007290dc in Csetjmp__ulongjmp (env=0x758270 <__libc_csu_init>, val=1) at ../src/C/Common/CsetjmpC.c:10 #3 0x0000000000720026 in RTExFrame__InvokeHandler (M3_BEcX3I_f=, M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:180 #4 0x000000000071fea2 in RTException__ResumeRaise (M3_Cblw37_a=) at ../src/runtime/ex_frame/RTExFrame.m3:148 #5 0x000000000071fcaf in RTException__Raise (M3_Cblw37_act=) at ../src/runtime/ex_frame/RTExFrame.m3:94 #6 0x0000000000704811 in RTHooks__Raise (M3_AJWxb1_ex=, M3_AJWxb1_arg=, M3_AJWxb1_module=, M3_AcxOUs_line=) at ../src/runtime/common/RTHooks.m3:79 #7 0x00000000006cacc4 in OSErrorPosix__Raise0 (M3_AcxOUs_errno=) at ../src/os/POSIX/OSErrorPosix.m3:50 #8 0x00000000006cc280 in FSPosix__Fail (M3_Bd56fi_p=, M3_DIYzyQ_f=) at ../src/os/POSIX/FSPosix.m3:359 #9 0x00000000006cc033 in FS__Status (M3_Bd56fi_pn=, M3_DKMpWi__result=) at ../src/os/POSIX/FSPosix.m3:328 #10 0x0000000000693831 in M3File__IsReadable (M3_Bd56fi_path=) at ../src/M3File.m3:123 #11 0x00000000004e5514 in MxConfig__TryConfig (M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/MxConfig.m3:122 #12 0x00000000004e4f12 in MxConfig__FindConfig () at ../src/MxConfig.m3:51 #13 0x00000000004e4c56 in MxConfig__FindFile () at ../src/MxConfig.m3:19 #14 0x0000000000436ed3 in Main__DoIt () at ../src/Main.m3:58 #15 0x0000000000437c74 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #16 0x0000000000713c81 in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x000000000071300c in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000007130a0 in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000403ae2 in main (argc=1, argv=0x7fffffff9bb8, envp=0x7fffffff9bc8) at _m3main.c:22 (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) On Sat, May 12, 2018 at 1:16 AM, Mika Nystrom > wrote: Hi m3devel, Jay, Tony, others, (Sorry for copying so many people but I am not sure m3devel is working..) I ran into some issues with m3core, which I eventually want to extend to support coroutines, but also has a bug (in Process.Create -- it doesn't check the group executable bits correctly and I wanted to fix that). So I thought I would do what I do every few years, which is update my CM3 to the latest available. Some years ago, Tony sent me instructions that I have been following, and here is what I do (and did): setenv CM3 `which cm3` # this is my old compiler, about five years old CM3, works well cd m3-sys/m3cc $CM3 # it logs "not shipping".. not sure what this means $CM3 -ship # I get a cm3cg in a subdirectory then I do cd ../../m3-libs/m3core rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship cd ../libm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../sysutils ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../../m3-sys/m3middle/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3objfile/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3back ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3linker/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3front ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../m3quake/ ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship ; cd ../cm3 ; rm -rf AMD64_LINUX ; $CM3 ; $CM3 -ship All this completes without errors. Yay! ... yay? Then I see that the new compiler gets shipped to the ship area (but not installed). Here, Tony's instructions are to setenv CM3 to the new compiler and repeat the above process. Then build the world a few times and see that all is well. But... sccj004186> /usr/intel/bin/gdb /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 GNU gdb (GDB) 7.11 ... Reading symbols from /p/hlp/utils/cm3/pkg/cm3/AMD64_LINUX/cm3...done. (gdb) run Starting program: /nfs/sc/disks/hlp_0002/utils/cm3/pkg/cm3/AMD64_LINUX/cm3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaabf6273e in __longjmp () from /lib64/libc.so.6 #1 0x6ea342bd1ed62ba2 in ?? () Backtrace stopped: Cannot access memory at address 0x5e7fec5d451375bc (gdb) Any ideas where I am going wrong? I could try to bootstrap from a binary dist, but I don't see why that would help. My machine is, yes, clearly AMD64_LINUX. Grateful for any assistance or ideas. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat May 12 16:53:28 2018 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 12 May 2018 10:53:28 -0400 Subject: [M3devel] release formats? In-Reply-To: References: Message-ID: <20180512145327.GA18564@topoi.pooq.com> On Sat, May 12, 2018 at 12:42:42PM +0000, Jay K wrote: > .tar.gz? > > tar.xz? > > .zip? > > .msi? > > .deb? (1) Is there one for the Raspberry Pi? (2) A tarred gzipped portable C archive would be nice for some users.. Even if there are different ones for 32 and 64 bit machines. -- hendrik From jayk123 at hotmail.com Mon May 14 10:43:43 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 14 May 2018 08:43:43 +0000 Subject: [M3devel] endianness again Message-ID: I'd like to for the system from m3core up to cm3 to generate the same C for all systems. It almost does. This enable C backend output as a distribution format. In order to get closer to that, I'd like bitfields removed from this part of the system. In particular in RT0.i3. To replace the bitfields with some amount of shifting and masking. Maybe ok? Alternatively..depending on interop needs, can we change the bitfield layout rules to be endian-independent? The problem with that is it hurts C interop, but arguably in an obscure area. Does anyone really use bitfields with <*external*> and depend on it working? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon May 14 20:14:22 2018 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 14 May 2018 13:14:22 -0500 Subject: [M3devel] endianness again In-Reply-To: References: Message-ID: <028fa07e-d250-becf-a50d-95d38c24d56a@lcwb.coop> On 05/14/2018 03:43 AM, Jay K wrote: > I'd like to for the system from m3core up to cm3 to generate the same C for all systems. > > It almost does. > > This enable C backend output as a distribution format. > > In order to get closer to that, I'd like bitfields removed from this part of the system. > > In particular in RT0.i3. > > To replace the bitfields with some amount of shifting and masking. > > > Maybe ok? You're talking about not *using* bitfields in the code of m3core, right? Sounds like a good idea to me, just in general. When doing bootstrappable* code, it is normal to avoid using certain language concepts to facilitate the process. I'd bet there are other language features unused in m3core, and maybe all the way up through the compiler, that would not bootstrap, but are, unfortunately not thus documented. Surely dependence on GC must be an example. *that one segfaulted my email client's spell checker :-). > > Alternatively..depending on interop needs, can we change the bitfield layout rules to be endian-independent? I have some unimplemented notes from long past on how to implement <*ENDIAN BIG*>, <*ENDIAN LITTLE*>, and <*ENDIAN NATIVE*> on records and arrays, but it has never gotten that close to the top of my priority list. The subject is absolutely mind-boggling. Understanding little endian really needs a person whose native language puts numeric data least-significant-first when embedded in textual information. I recently got the impression Hebrew is this way, but not sure. I find mere right-to-left reading is not so difficult. > The problem with that is it hurts C interop, but arguably in an obscure area. I am currently working on better compiler support of bitfields. But don't panic. It's all in the front end, ahead of M3CG_Ops.i3. It would not help the portability problem you are talking about. > Does anyone really use bitfields with <*external*> and depend on it working? Not I. > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Mon May 14 20:30:40 2018 From: lists at darko.org (Darko Volaric) Date: Mon, 14 May 2018 20:30:40 +0200 Subject: [M3devel] endianness again In-Reply-To: References: Message-ID: So what I'm guessing is you're proposing that instead of the compiler figuring out the bit operations for extracting bit fields you're going to do them yourself? That sounds like a terrible idea and likely to introduce bugs that are hard to identify and fix and well as code that is hard to maintain. I also think generating C is a terrible idea. Why are we hacking the compiler to support this misfeature? If it's to support portability then I think it's the wrong approach and not likely to actually improve portability since C is less portable than M3. The current structure of the compiler and libraries is at fault here wrt portability. I don't know about other users, but I use M3 becuase it's not C. I'd like to move the compiler toward having no C sources and no C generation. On Mon, May 14, 2018 at 10:43 AM, Jay K wrote: > I'd like to for the system from m3core up to cm3 to generate the same C > for all systems. > > It almost does. > This enable C backend output as a distribution format. > > In order to get closer to that, I'd like bitfields removed from this part > of the system. > > In particular in RT0.i3. > > To replace the bitfields with some amount of shifting and masking. > > Maybe ok? > > Alternatively..depending on interop needs, can we change the bitfield > layout rules to be endian-independent? > The problem with that is it hurts C interop, but arguably in an obscure > area. > Does anyone really use bitfields with <*external*> and depend on it > working? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon May 14 21:41:52 2018 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 14 May 2018 19:41:52 +0000 (UTC) Subject: [M3devel] endianness again In-Reply-To: References: Message-ID: <392859553.2093358.1526326912761@mail.yahoo.com> Hello: I thought portability problems, were solved long ago (crossassembling VAX to micro) by JVM; and we have the RT adapted to it in JVM CM implementation. Of course we need to adapt the compiler to emit JVM bytecode; but in any case; JVM has a lot of opportunities of optimization; I was recently reviewing this article [1] Meanwhile executing fast C is not as easy as much [2] However Critical Mass JVM was designed from scratch to support interoperability with C; so we need to be careful to be kind for both languages to mantain its functionality Thanks in advance [1]?Writing concurrent assemblers?a case study in path pascal - Dowsing - 1986 - Software: Practice and Experience - Wiley Online Library?. [On line]. Disponible en: https://onlinelibrary.wiley.com/doi/pdf/10.1002/spe.4380161206. [Accesed: 14-may-2018]. [2]?Automatic Parallelization of C by Means of Language Transcription | SpringerLink?. [On line]. Disponible en: https://link.springer.com/chapter/10.1007/3-540-48319-5_11. [Accesed: 14-may-2018]. ? En lunes, 14 de mayo de 2018 13:31:01 GMT-5, Darko Volaric escribi?: So what I'm guessing is you're proposing that instead of the compiler figuring out the bit operations for extracting bit fields you're going to do them yourself? That sounds like a terrible idea and likely to introduce bugs that are hard to identify and fix and well as code that is hard to maintain. I also think generating C is a terrible idea. Why are we hacking the compiler to support this misfeature? If it's to support portability then I think it's the wrong approach and not likely to actually improve portability since C is less portable than M3. The current structure of the compiler and libraries is at fault here wrt portability. I don't know about other users, but I use M3 becuase it's not C. I'd like to move the compiler toward having no C sources and no C generation. On Mon, May 14, 2018 at 10:43 AM, Jay K wrote: I'd like to for the system from m3core up to?cm3 to generate the same C for all systems. It almost does. This enable C backend output as a distribution format. In order to get closer to that, I'd like bitfields removed from this part of the system. In particular in RT0.i3. To replace the bitfields with some amount of shifting and?masking. Maybe ok? Alternatively..depending on interop needs, can we? change the bitfield layout rules to be endian-independent?The problem with that is it?hurts C interop, but arguably in an obscure area.Does anyone really use bitfields with? <*external*>? and depend on it working? ?- Jay ______________________________ _________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/ mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon May 14 22:17:45 2018 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 14 May 2018 20:17:45 +0000 (UTC) Subject: [M3devel] endianness again In-Reply-To: <392859553.2093358.1526326912761@mail.yahoo.com> References: <392859553.2093358.1526326912761@mail.yahoo.com> Message-ID: <846667938.2123106.1526329065610@mail.yahoo.com> Hello again: just my two cents: Java lack of unsafe construct annoyed programmers in real world, so CM tried to fix that, as it's explained nicely here imn last parragraph [1] Needless to say to say PDP11/VAX11 was the memory model for current C. [1]??Re: [Cryptography] bounded pointers in C? - MARC?. [On line]. Available onn: https://marc.info/?l=cryptography&m=139784275512662. [Accesed: 14-may-2018]. En lunes, 14 de mayo de 2018 14:41:52 GMT-5, Daniel Alejandro Benavides D. escribi?: Hello: I thought portability problems, were solved long ago (crossassembling VAX to micro) by JVM; and we have the RT adapted to it in JVM CM implementation. Of course we need to adapt the compiler to emit JVM bytecode; but in any case; JVM has a lot of opportunities of optimization; I was recently reviewing this article [1] Meanwhile executing fast C is not as easy as much [2] However Critical Mass JVM was designed from scratch to support interoperability with C; so we need to be careful to be kind for both languages to mantain its functionality Thanks in advance [1]?Writing concurrent assemblers?a case study in path pascal - Dowsing - 1986 - Software: Practice and Experience - Wiley Online Library?. [On line]. Disponible en: https://onlinelibrary.wiley.com/doi/pdf/10.1002/spe.4380161206. [Accesed: 14-may-2018]. [2]?Automatic Parallelization of C by Means of Language Transcription | SpringerLink?. [On line]. Disponible en: https://link.springer.com/chapter/10.1007/3-540-48319-5_11. [Accesed: 14-may-2018]. ? En lunes, 14 de mayo de 2018 13:31:01 GMT-5, Darko Volaric escribi?: So what I'm guessing is you're proposing that instead of the compiler figuring out the bit operations for extracting bit fields you're going to do them yourself? That sounds like a terrible idea and likely to introduce bugs that are hard to identify and fix and well as code that is hard to maintain. I also think generating C is a terrible idea. Why are we hacking the compiler to support this misfeature? If it's to support portability then I think it's the wrong approach and not likely to actually improve portability since C is less portable than M3. The current structure of the compiler and libraries is at fault here wrt portability. I don't know about other users, but I use M3 becuase it's not C. I'd like to move the compiler toward having no C sources and no C generation. On Mon, May 14, 2018 at 10:43 AM, Jay K wrote: I'd like to for the system from m3core up to?cm3 to generate the same C for all systems. It almost does. This enable C backend output as a distribution format. In order to get closer to that, I'd like bitfields removed from this part of the system. In particular in RT0.i3. To replace the bitfields with some amount of shifting and?masking. Maybe ok? Alternatively..depending on interop needs, can we? change the bitfield layout rules to be endian-independent?The problem with that is it?hurts C interop, but arguably in an obscure area.Does anyone really use bitfields with? <*external*>? and depend on it working? ?- Jay ______________________________ _________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/ mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon May 14 23:00:18 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 14 May 2018 21:00:18 +0000 Subject: [M3devel] endianness again In-Reply-To: References: , Message-ID: We can already build most of the system by converting to C. Not idiomatic C, but very wierd looking C. This has enabled an AMD64_NT port, I believe ARM_LINUX, and I'll do ARM64_LINUX hopefully soon. It makes the system much much portable, and cuts the dependency on gcc, and LLVM. It enables not using an old gcc. It enables a level of portability of the system approaching helloworld. It increases debugability of the system where m3gdb doesn't run -- Mac and Windows. Can this can be further improved. C and C++ really are very portable. I don't understand why people claim otherwise. There are millions of lines of code that are OS-portable and/or architecture-portable in C and C++. Far more than Modula-3. Yes I'd use shifting and masking and it would be correct and bugfree. I'd use the Word interface. Nothing much is at fault. If fault must be found, it is in the tension between self-written languages is good for understanding and proving the language, and bad for bootstrapping. People struggle to build our system. This would provide a step forward the typical: tar xf cm3.tar.gz. cd cm3 ./configure (not yet) make (already, but only builds cm3) This already works, but needs work to build the entire system, beyond cm3. By removing the endian-ness dependency, the distributed cm3.tar.gz can be one and the same for more systems, i.e. sparc and powerpc and x86. Beyond that, there is word size and NT/Posix dependency. - Jay ________________________________ From: Darko Volaric Sent: Monday, May 14, 2018 6:30 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] endianness again So what I'm guessing is you're proposing that instead of the compiler figuring out the bit operations for extracting bit fields you're going to do them yourself? That sounds like a terrible idea and likely to introduce bugs that are hard to identify and fix and well as code that is hard to maintain. I also think generating C is a terrible idea. Why are we hacking the compiler to support this misfeature? If it's to support portability then I think it's the wrong approach and not likely to actually improve portability since C is less portable than M3. The current structure of the compiler and libraries is at fault here wrt portability. I don't know about other users, but I use M3 becuase it's not C. I'd like to move the compiler toward having no C sources and no C generation. On Mon, May 14, 2018 at 10:43 AM, Jay K > wrote: I'd like to for the system from m3core up to cm3 to generate the same C for all systems. It almost does. This enable C backend output as a distribution format. In order to get closer to that, I'd like bitfields removed from this part of the system. In particular in RT0.i3. To replace the bitfields with some amount of shifting and masking. Maybe ok? Alternatively..depending on interop needs, can we change the bitfield layout rules to be endian-independent? The problem with that is it hurts C interop, but arguably in an obscure area. Does anyone really use bitfields with <*external*> and depend on it working? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue May 15 00:04:46 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 14 May 2018 22:04:46 +0000 Subject: [M3devel] git workflow Message-ID: Let's please use squash commits or whatever they are called, not merge commits. i.e. with descriptions, not "merged from branch". Also, I'd like it to work, where a PR is approved, and then the author can merge/squash it. This gives him a chance to rewrite the commit message. i.e. this looks bad: https://github.com/modula3/cm3/commits/master this looks better: https://github.com/jaykrell/cm3/commits/master - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue May 15 00:12:21 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 14 May 2018 22:12:21 +0000 Subject: [M3devel] endianness again In-Reply-To: <028fa07e-d250-becf-a50d-95d38c24d56a@lcwb.coop> References: , <028fa07e-d250-becf-a50d-95d38c24d56a@lcwb.coop> Message-ID: > I have some unimplemented notes from long past on how to > implement <*ENDIAN BIG*>, > <*ENDIAN LITTLE*>, and <*ENDIAN NATIVE*> on records and arrays, but it has never You are just avoiding the C preprocessor, or a runtime check. Here is how half the world does it: https://github.com/mono/mono/blob/master/mono/metadata/decimal-ms.c#L87 I'd want this conditionality to flow all the way to the backend. The frontend shouldn't have to know. The C backend would generate both combinations and put ifdef around. Leaving portable C code. m3cc would just pick the right one. Ultimately for now I'd just push the code into ifdefed C code. Like where we pick apart floats/doubles. Or a runtime check. - Jay ________________________________ From: Rodney M. Bates Sent: Monday, May 14, 2018 6:14 PM To: Jay K; m3devel Subject: Re: [M3devel] endianness again On 05/14/2018 03:43 AM, Jay K wrote: > I'd like to for the system from m3core up to cm3 to generate the same C for all systems. > > It almost does. > > This enable C backend output as a distribution format. > > In order to get closer to that, I'd like bitfields removed from this part of the system. > > In particular in RT0.i3. > > To replace the bitfields with some amount of shifting and masking. > > > Maybe ok? You're talking about not *using* bitfields in the code of m3core, right? Sounds like a good idea to me, just in general. When doing bootstrappable* code, it is normal to avoid using certain language concepts to facilitate the process. I'd bet there are other language features unused in m3core, and maybe all the way up through the compiler, that would not bootstrap, but are, unfortunately not thus documented. Surely dependence on GC must be an example. *that one segfaulted my email client's spell checker :-). > > Alternatively..depending on interop needs, can we change the bitfield layout rules to be endian-independent? I have some unimplemented notes from long past on how to implement <*ENDIAN BIG*>, <*ENDIAN LITTLE*>, and <*ENDIAN NATIVE*> on records and arrays, but it has never gotten that close to the top of my priority list. The subject is absolutely mind-boggling. Understanding little endian really needs a person whose native language puts numeric data least-significant-first when embedded in textual information. I recently got the impression Hebrew is this way, but not sure. I find mere right-to-left reading is not so difficult. > The problem with that is it hurts C interop, but arguably in an obscure area. I am currently working on better compiler support of bitfields. But don't panic. It's all in the front end, ahead of M3CG_Ops.i3. It would not help the portability problem you are talking about. > Does anyone really use bitfields with <*external*> and depend on it working? Not I. > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue May 15 10:19:10 2018 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 15 May 2018 01:19:10 -0700 Subject: [M3devel] this is the old thread re: longrealtype, realtype Message-ID: <20180515081910.929DC1A2069@async.async.caltech.edu> ------- Forwarded Message Return-Path: jayk123 at hotmail.com Delivery-Date: Tue Jul 20 05:28:41 2010 Return-Path: X-Original-To: mika at async.caltech.edu Delivered-To: mika at async.caltech.edu Received: by async.async.caltech.edu (Postfix, from userid 1010) id 0DE081A2091; Tue, 20 Jul 2010 05:28:41 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on async.async.caltech.edu X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.4 Received: from col0-omc3-s14.col0.hotmail.com (col0-omc3-s14.col0.hotmail.com [65.55.34.152]) by async.async.caltech.edu (Postfix) with ESMTP id CE6851A2080 for ; Tue, 20 Jul 2010 05:28:38 -0700 (PDT) Received: from COL101-W12 ([65.55.34.137]) by col0-omc3-s14.col0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 20 Jul 2010 05:28:37 -0700 Message-ID: X-Originating-IP: [71.112.27.90] From: Jay K Sender: To: m3devel CC: Mika Nystrom Subject: RE: LongrealType missed, was: Re: [M3devel] "FreeBSD4" false advertising Date: Tue, 20 Jul 2010 12:28:36 +0000 Importance: Normal In-Reply-To: References: <20100714094120.93B001A2098 at async.async.caltech.edu>,<20100714114648.nhnwruh008840wso at mail.elegosoft.com>,<20100714100138.631581A2098 at async.async.caltech.edu>,<20100714100525.BEE931A2096 at async.async.caltech.edu>,,<20100714104312.155351A2098 at async.async.caltech.edu>,<20100714125516.1mzfehgnkgcs800w at mail.elegosoft.com>, Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 20 Jul 2010 12:28:37.0063 (UTC) FILETIME=[13B3E570:01CB2807] Mika=2C did this work for you? =A0- Jay - ---------------------------------------- > From: jay.krell at cornell.edu > To: wagner at elegosoft.com=3B m3devel at elegosoft.com > CC: mika at async.caltech.edu > Subject: RE: LongrealType missed=2C was: Re: [M3devel] "FreeBSD4" false a= dvertising > Date: Wed=2C 14 Jul 2010 11:23:49 +0000 > > > Mika=2C Please try updating libm3 and see if that works for you. Thanks. > > - Jay > > ---------------------------------------- > > Date: Wed=2C 14 Jul 2010 12:55:16 +0200 > > From: wagner at elegosoft.com > > To: m3devel at elegosoft.com > > CC: jay.krell at cornell.edu=3B mika at async.caltech.edu > > Subject: LongrealType missed=2C was: Re: [M3devel] "FreeBSD4" false adv= ertising > > > > Quoting Mika Nystrom : > > > > > Argh is it really necessary to break source compatibility here? > > > > > > I also don't like that I as a client have to import IEEE things when = all > > > I want is "LONGREAL" (and let Modula-3 do whatever it wants with that= ). > > > > > > I would propose at least having an interface LongrealType with > > > > > > CONST Hash =3D Longreal.Hash > > > > > > etc. so as not to force clients to import all the nitty-gritty about > > > various floating point representations. And so as not to break source > > > compatibility! If I change this by removing Type=2C my code will no l= onger > > > compile with the old compilers.... > > > > Hm=2C I'm a little bit out of context here... > > What was the reason for this change? It seems nobody else has bothered > > so far. > > > > This was the commit in question: > > > > 2009-12-23 21:43 hosking > > > > * m3-libs/: m3core/src/m3makefile=2C libm3/src/types/ASCII.i3=2C > > libm3/src/types/ASCII.m3=2C libm3/src/types/Boolean.i3=2C > > libm3/src/types/Boolean.m3=2C libm3/src/types/COPYRIGHT-CMASS=2C > > libm3/src/types/Char.i3=2C libm3/src/types/Char.m3=2C > > libm3/src/types/Int32.i3=2C libm3/src/types/Int32.m3=2C > > libm3/src/types/Int64.i3=2C libm3/src/types/Int64.m3=2C > > libm3/src/types/Integer.i3=2C libm3/src/types/Integer.m3=2C > > libm3/src/types/Longint.i3=2C libm3/src/types/Longint.m3=2C > > libm3/src/types/LongrealType.i3=2C libm3/src/types/LongrealType.m3=2C > > libm3/src/types/RealType.i3=2C libm3/src/types/RealType.m3=2C > > libm3/src/types/Refany.i3=2C libm3/src/types/Refany.m3=2C > > libm3/src/types/Unicode.i3=2C libm3/src/types/Unicode.m3=2C > > libm3/src/types/WideChar.i3=2C libm3/src/types/WideChar.m3=2C > > libm3/src/types/m3makefile=2C m3core/src/float/Common/Extended.m3=2C > > m3core/src/float/Common/LongReal.m3=2C m3core/src/float/Common/Real.m3= =2C > > m3core/src/float/IEEE/Extended.i3=2C m3core/src/float/IEEE/LongReal.i3= =2C > > m3core/src/float/IEEE/Real.i3=2C m3core/src/float/VAX/Extended.i3=2C > > m3core/src/float/VAX/LongReal.i3=2C m3core/src/float/VAX/Real.i3: > > > > Move libm3/src/types into m3core. > > Note that LongrealType and RealType have been folded into m3core/src/fl= oat. > > Clients will need adjustment. > > > > Any comments? > > > > Olaf > > > > > Mika > > > > > > Jay K writes: > > >> > > >> Remove "Type" it appears. > > >> > > >> > > >> http://modula3.elegosoft.com/cm3/ChangeLog-2009 > > >> > > >> search for "clients will need". > > >> > > >> e.g. > > >> > > >> +++ cm3/m3-demo/mentor/src/binpack/m3makefile 2009/12/23 22:05:06 1.= 2 > > >> @@ -10=3D2C7 +10=3D2C7 @@ > > >> =3D20 > > >> import ("zeus") > > >> =3D20 > > >> -list ("Real"=3D2C "RealType") > > >> +list ("Real"=3D2C "Real") > > >> =3D20 > > >> zume ("Binpack") > > >> oblume ("Binpack"=3D2C "myview") > > >> > > >> --- cm3/m3-demo/mentor/src/binpack/RealList.i3 2001/01/13 14:18:20 1= .1 > > >> +++ cm3/m3-demo/mentor/src/binpack/RealList.i3 2009/12/23 22:05:05 1= .2 > > >> @@ -2=3D2C4 +2=3D2C4 @@ > > >> (* Distributed only by permission. *) > > >> (* Last modified on Fri Jul 9 16:36:47 PDT 1993 by mhb *) > > >> =3D20 > > >> -INTERFACE RealList =3D3D List(RealType) END RealList. > > >> +INTERFACE RealList =3D3D List(Real) END RealList. > > >> > > >> --- cm3/m3-demo/mentor/src/binpack/RealList.m3 2001/01/13 14:18:20 1= .1 > > >> +++ cm3/m3-demo/mentor/src/binpack/RealList.m3 2009/12/23 22:05:06 1= .2 > > >> @@ -2=3D2C4 +2=3D2C4 @@ > > >> (* All rights reserved. *) > > >> (* See the file COPYRIGHT for a full description. *) > > >> =3D20 > > >> -MODULE RealList =3D3D List(RealType) END RealList. > > >> +MODULE RealList =3D3D List(Real) END RealList. > > >> > > >> =3DA0- Jay > > >> > > >> ---------------------------------------- > > >>> To: wagner at elegosoft.com > > >>> CC: jay.krell at cornell.edu=3D3B mika at async.caltech.edu > > >>> Subject: Re: [M3devel] "FreeBSD4" false advertising > > >>> Date: Wed=3D2C 14 Jul 2010 03:05:25 -0700 > > >>> From: mika at async.async.caltech.edu > > >>> > > >>> I'm not actually sure I understand the intent of the changes in thi= s area=3D > > >> . > > >>> > > >>> My old m3makefile has: > > >>> > > >>> Table("TextLongReal"=3D2C "Text"=3D2C "LongrealType") > > >>> > > >>> What am I supposed to use now? > > >>> > > >>> Mika > > >>> > > >>> Mika Nystrom writes: > > >>> >Olaf Wagner writes: > > >>> >>Quoting Mika Nystrom : > > >>> >> > > >>> >>> Hi Jay=3D2C Olaf=3D2C > > >>> >>> > > >>> >>> I actually built everything! After backing out some of my chang= es > > >>> >>> that I had tried to get 5.8.6-REL to build with=3D2C it worked= =3D2C and m=3D > > >> entor > > >>> >>> ran=3D2C even. > > >>> >> > > >>> >>Great! So it's still possible to build cm3 on FreeBSD 4=3D2C thou= gh > > >>> >>not our release code. > > >>> > > > >>> >The differences relative to the current CVS head are very minor. > > >>> > > > >>> >1. Added FreeBSD4 to thread.quake as a non-pthread platform > > >>> > > > >>> >2. got rid of the -m32 flags in FreeBSD4.conf > > >>> > > > >>> >Everything else that needed to change I think Jay has managed to p= ut in > > >>> >the CVS head. > > >>> > > > >>> >> > > >>> >>> But... what happened to libm3/src/types? Building my own code I= get > > >>> >>> the following errors: > > >>> >>> > > >>> >>> new source -> compiling LRInterval.i3 > > >>> >>> "../FreeBSD4/LRInterval.i3 =3D3D3D> ../src/Interval.ig"=3D2C li= ne 5: unab=3D > > >> le to =3D3D20 > > >>> >>> find interface (LongrealType) > > >>> >>> 1 error encountered > > >>> >>> > > >>> >>> I can't find LongrealType anywhere=3D2C do I have a broken chec= kout? We=3D > > >> ll=3D2C it > > >>> >>> is in one place=3D2C in "doc". (Same for RealType=3D2C maybe ot= hers?) > > >>> >> > > >>> >>See =3D3D20 > > >>> >>http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/libm3= /src/t=3D > > >> ypes/=3D3D > > >>> >>Attic/LongrealType.i3 > > >>> > > > >>> >Surely this is wrong?? I am looking for an interface that has T = =3D3D > > >>> >LONGREAL=3D2C not something that lets me muck around with the repr= esentati=3D > > >> on > > >>> >of IEEE=3D2C VAX=3D2C etc.=3D2C floating-point formats. > > >>> > > > >>> >Why is it even in m3core? This seems clearly like libm3 stuff.. I = notice=3D > > >> d > > >>> >Boolean.i3 also moved. > > >>> > > > >>> >Also if there is no importable interface called LongrealType that = is goi=3D > > >> ng > > >>> >to cause endless problems with source that is supposed to compile = under > > >>> >different versions of Modula-3. Even relatively recent CM3s requir= ed > > >>> >you to use LongrealType to instantiate generics. > > >>> > > > >>> > Mika > > >> =3D > > > > > > > > > > > -- > > Olaf Wagner -- elego Software Solutions GmbH > > Gustav-Meyer-Allee 25 / Geb=E4ude 12=2C 13355 Berlin=2C Germany > > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86= 95 > > http://www.elegosoft.com | Gesch=E4ftsf=FChrer: Olaf Wagner | Sitz: Ber= lin > > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE1632= 14194 > > > = ------- End of Forwarded Message From jayk123 at hotmail.com Tue May 15 23:11:44 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 21:11:44 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: Message-ID: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue May 15 23:13:05 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 21:13:05 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , Message-ID: > undefined reference to `__gxx_personality_v0' This is from compiling as C++ but linking as C. Make them match, or compile as C but link as C++. - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, May 15, 2018 9:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Not that much at a loss. The following diff resolves things for me, also enables the parallel back end as well as defines the libraries I was ineptly pulling in before. But does it break anything for the compiler? diff -cr cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/cm3.cfg /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/cm3.cfg *** cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/cm3.cfg Mon May 14 23:10:39 2018 --- /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/cm3.cfg Tue May 15 01:57:26 2018 *************** *** 2,5 **** if not defined("HOST") HOST = "AMD64_LINUX" end if not defined("TARGET") TARGET = HOST end INSTALL_ROOT = (path() & SL & "..") ! include(path() & SL & "config" & SL & TARGET) \ No newline at end of file --- 2,6 ---- if not defined("HOST") HOST = "AMD64_LINUX" end if not defined("TARGET") TARGET = HOST end INSTALL_ROOT = (path() & SL & "..") ! include(path() & SL & "config" & SL & TARGET) ! M3_PARALLEL_BACK=30 diff -cr cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/AMD64_LINUX /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/AMD64_LINUX *** cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/AMD64_LINUX Mon May 14 23:10:39 2018 --- /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/AMD64_LINUX Tue May 15 02:08:39 2018 *************** *** 1,8 **** readonly TARGET = "AMD64_LINUX" % code generation target readonly GNU_PLATFORM = "amd64-linux" % "cpu-os" string for GNU ! SYSTEM_CC = "g++ -gstabs+ -m64 -fPIC" % C compiler readonly SYSTEM_ASM = "as --64" % Assembler include("AMD64.common") include("Linux.common") --- 1,10 ---- readonly TARGET = "AMD64_LINUX" % code generation target readonly GNU_PLATFORM = "amd64-linux" % "cpu-os" string for GNU ! SYSTEM_CXXC = "g++ -gstabs+ -m64 -fPIC" % C compiler ! SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" % C compiler readonly SYSTEM_ASM = "as --64" % Assembler + PTHREADS="yes" include("AMD64.common") include("Linux.common") diff -cr cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Linux.common /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Linux.common *** cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Linux.common Mon May 14 23:10:39 2018 --- /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Linux.common Tue May 15 02:11:45 2018 *************** *** 5,11 **** readonly TARGET_OS = "LINUX" ! SYSTEM_LD = SYSTEM_CC & " -Wl,-z,now" & " -Wl,-z,origin" & " -Bsymbolic" --- 5,11 ---- readonly TARGET_OS = "LINUX" ! SYSTEM_LD = SYSTEM_CXXC & " -Wl,-z,now" & " -Wl,-z,origin" & " -Bsymbolic" diff -cr cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Unix.common /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Unix.common *** cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Unix.common Mon May 14 23:10:39 2018 --- /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/bin/config/Unix.common Tue May 15 02:29:34 2018 *************** *** 47,53 **** % "ODBC" : [ "-lodbc" ], "MOTIF" : [ "-lXm" ], "TCP" : [ ], ! "X11" : [ "-L/usr/X11R6/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ] } % See Linux.common, FreeBSD.common, OpenBSD.common, NetBSD.common, etc. --- 47,55 ---- % "ODBC" : [ "-lodbc" ], "MOTIF" : [ "-lXm" ], "TCP" : [ ], ! "X11" : [ "-L/usr/X11R6/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ], ! "READLINE" : [ "-lreadline" ], ! "EXPAT" : [ "-lexpat" ] } % See Linux.common, FreeBSD.common, OpenBSD.common, NetBSD.common, etc. *************** *** 63,72 **** if not defined ("SYSTEM_LIBORDER") if HasOdbc() SYSTEM_LIBORDER = [ "OPENGL", "DECPEX", "MOTIF", "X11", "TCP", ! "ODBC", "POSTGRES95", "FLEX-BISON", "LEX-YACC", "LIBC" ] else SYSTEM_LIBORDER = [ "OPENGL", "DECPEX", "MOTIF", "X11", "TCP", ! "POSTGRES95", "FLEX-BISON", "LEX-YACC", "LIBC" ] end end --- 65,74 ---- if not defined ("SYSTEM_LIBORDER") if HasOdbc() SYSTEM_LIBORDER = [ "OPENGL", "DECPEX", "MOTIF", "X11", "TCP", ! "ODBC", "POSTGRES95", "FLEX-BISON", "LEX-YACC", "READLINE", "EXPAT", "LIBC" ] else SYSTEM_LIBORDER = [ "OPENGL", "DECPEX", "MOTIF", "X11", "TCP", ! "POSTGRES95", "FLEX-BISON", "LEX-YACC", "READLINE", "EXPAT", "LIBC" ] end end *************** *** 226,232 **** local args = options if M3_PROFILING args += "-pg" end if debug args += "-g" end ! local ret_code = try_exec (SYSTEM_CC, args, "-xc++ -c", source, "-o", object) if not equal(ret_code, 0) and equal(source, "_m3main.c") % for _m3main.c, try also as C instead of C++ for older cm3 % NOTE: This is gcc specific. --- 228,241 ---- local args = options if M3_PROFILING args += "-pg" end if debug args += "-g" end ! ! local ret_code = 1 ! ! if equal(source, "_m3main.c") ! ret_code = try_exec (SYSTEM_CXXC, args, "-c -xc++", source, "-o", object) ! else ! ret_code = try_exec (SYSTEM_CC, args, "-c", source, "-o", object) ! end if not equal(ret_code, 0) and equal(source, "_m3main.c") % for _m3main.c, try also as C instead of C++ for older cm3 % NOTE: This is gcc specific. On Tue, May 15, 2018 at 2:01 AM, Mika Nystrom > wrote: OK I guess I can't just change it back to use the C compiler because that breaks the compilation of _m3main.c new source -> compiling Main.m3 **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 30 -> linking m3tmplhack gcc -gstabs+ -m64 -fPIC -g -c _m3main.c -o _m3main.o /tmp/cm3-all-AMD64_LINUX-d5.10.0-Linux3.0.101-20180514/pkg/m3core/AMD64_LINUX/libm3core.a(ThreadPThreadC.o): In function `SignalHandlerC': /nfs/sc/disks/hlp_0015/mnystroe/git/cm3/m3-libs/m3core/AMD64_LINUX/../src/thread/PTHREAD/ThreadPThreadC.c:87: undefined reference to `__gxx_personality_v0' Now I'm at a loss. On Tue, May 15, 2018 at 1:41 AM, Mika Nystrom > wrote: So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue May 15 23:16:10 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 21:16:10 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: Message-ID: I put: #ifdef __cplusplus extern "C" { #endif ... #ifdef __cplusplus } #endif around things. There is another way not simple. Upon certain link errors we could produce C wrappers that call the C++ functions (via another layer with different names), or each the Modula-3 compiler to mangle (platform-specific), optionally depending on errors. Or linkers could do it. but I guess we can have cplusplus_source. I guess I was being lazy not to have it. - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue May 15 23:17:45 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 21:17:45 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , Message-ID: e.g. https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/unix/Common/Uexec.c#L8 ________________________________ From: Jay K Sent: Tuesday, May 15, 2018 9:16 PM To: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 I put: #ifdef __cplusplus extern "C" { #endif ... #ifdef __cplusplus } #endif around things. There is another way not simple. Upon certain link errors we could produce C wrappers that call the C++ functions (via another layer with different names), or each the Modula-3 compiler to mangle (platform-specific), optionally depending on errors. Or linkers could do it. but I guess we can have cplusplus_source. I guess I was being lazy not to have it. - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue May 15 23:54:01 2018 From: lists at darko.org (Darko Volaric) Date: Tue, 15 May 2018 23:54:01 +0200 Subject: [M3devel] endianness again In-Reply-To: References: Message-ID: <2280BD91-27CD-4BF9-AD8A-667789E097F4@darko.org> Your claim about the portability of C sort of epitomizes what you miss about M3. When you say C is portable you mean that you can take C code and get it to compile on another platform and even get it to work correctly, if you apply enough #ifdefs, or it might work out of the box. When I say M3 is portable I mean that when you change platforms the code cannot work incorrectly, assuming you don't use unsafe features. That's the beauty of M3. I don't see how C removes dependencies on LLVM or GCC, it just adds another poorly defined interface - the C language - between M3 and those back ends. > On May 14, 2018, at 11:00 PM, Jay K wrote: > > We can already build most of the system by converting to C. > Not idiomatic C, but very wierd looking C. > This has enabled an AMD64_NT port, I believe ARM_LINUX, and I'll do ARM64_LINUX hopefully soon. > > It makes the system much much portable, and cuts the dependency on gcc, and LLVM. > It enables not using an old gcc. > > It enables a level of portability of the system approaching helloworld. > It increases debugability of the system where m3gdb doesn't run -- Mac and Windows. > Can this can be further improved. > > C and C++ really are very portable. I don't understand why people claim otherwise. > There are millions of lines of code that are OS-portable and/or architecture-portable in C and C++. > Far more than Modula-3. > > Yes I'd use shifting and masking and it would be correct and bugfree. > I'd use the Word interface. > > Nothing much is at fault. > If fault must be found, it is in the tension between self-written languages is good > for understanding and proving the language, and bad for bootstrapping. > People struggle to build our system. > This would provide a step forward the typical: > tar xf cm3.tar.gz. > cd cm3 > ./configure (not yet) > make (already, but only builds cm3) > > This already works, but needs work to build the entire system, beyond cm3. > > By removing the endian-ness dependency, the distributed cm3.tar.gz can be one and the same for more systems, i.e. sparc and powerpc and x86. > Beyond that, there is word size and NT/Posix dependency. > > - Jay > > From: Darko Volaric > > Sent: Monday, May 14, 2018 6:30 PM > To: Jay K > Cc: m3devel > Subject: Re: [M3devel] endianness again > > So what I'm guessing is you're proposing that instead of the compiler figuring out the bit operations for extracting bit fields you're going to do them yourself? That sounds like a terrible idea and likely to introduce bugs that are hard to identify and fix and well as code that is hard to maintain. > > I also think generating C is a terrible idea. Why are we hacking the compiler to support this misfeature? If it's to support portability then I think it's the wrong approach and not likely to actually improve portability since C is less portable than M3. The current structure of the compiler and libraries is at fault here wrt portability. > > I don't know about other users, but I use M3 becuase it's not C. I'd like to move the compiler toward having no C sources and no C generation. > > > On Mon, May 14, 2018 at 10:43 AM, Jay K > wrote: > I'd like to for the system from m3core up to cm3 to generate the same C for all systems. > It almost does. > This enable C backend output as a distribution format. > In order to get closer to that, I'd like bitfields removed from this part of the system. > In particular in RT0.i3. > To replace the bitfields with some amount of shifting and masking. > > Maybe ok? > > Alternatively..depending on interop needs, can we change the bitfield layout rules to be endian-independent? > The problem with that is it hurts C interop, but arguably in an obscure area. > Does anyone really use bitfields with <*external*> and depend on it working? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 00:19:40 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 22:19:40 +0000 Subject: [M3devel] endianness again In-Reply-To: <2280BD91-27CD-4BF9-AD8A-667789E097F4@darko.org> References: , <2280BD91-27CD-4BF9-AD8A-667789E097F4@darko.org> Message-ID: ifdefs are for, like, Unix vs. NT. Modula-3 has the same thing. Otherwise many many lines of code in either system are portable. Safety/unsafety isn't the defining characteristic of portability, but it is an important feature of Modula-3, esp. its optionality. You start with: int main() { printf("hello world\n"); return 0; } Very portable. Next you say free(malloc()) or std::vector().resize. Very very portable. Next you might say: pthread_create or CreateThread or std::thread_something. Last is very portable. First is same boat as Modula-3, but we already did it for you. Next you might want malloc() but no free(). Then you do have a problem. Again something Modula-3 handles for you, but I wouldn't really label it portability. It is part of safety. Not portability. It is true if send the C code through gcc, you are dependent on gcc. If you send it through clang, you are dependent on clang. Which interface to a C compiler would you prefer? gcc-specific "tree"; not standard, not easily moved between versions, but efficient LLVM, ditto C/C++ text, very well specified, works across a large variety -- I have had this working for years. I used it for AMD64_NT and I think ARM_LINUX. I'll do ARM64_LINUX soon. It interfaced successfully with the SunPro compiler also. You can look at records debugging on NT and Darwin, which you cannot do with the integrated or gcc backends (on those platforms). Definitely not as efficient at compile-time. Some things missing, like the other divide modes and tailcalls. Somewhat redistributable. The redistributableness is a goal I have here: tar xf configure make make install Use the C command line toolset as a portability vehicle, instead of the gcc source. Hopefully with C++ we'll also get efficient exception handling, way beyond what we have today. We probably need to churn the frontend and M3CG_Ops. Actually another idea C is a backstop and then we iterate on the integrated backend. But with a fallback, if that fails, or doesn't cover some architectures like alpha/mips/sparc/powerpc. - Jay ________________________________ From: Darko Volaric Sent: Tuesday, May 15, 2018 9:54 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] endianness again Your claim about the portability of C sort of epitomizes what you miss about M3. When you say C is portable you mean that you can take C code and get it to compile on another platform and even get it to work correctly, if you apply enough #ifdefs, or it might work out of the box. When I say M3 is portable I mean that when you change platforms the code cannot work incorrectly, assuming you don't use unsafe features. That's the beauty of M3. I don't see how C removes dependencies on LLVM or GCC, it just adds another poorly defined interface - the C language - between M3 and those back ends. On May 14, 2018, at 11:00 PM, Jay K > wrote: We can already build most of the system by converting to C. Not idiomatic C, but very wierd looking C. This has enabled an AMD64_NT port, I believe ARM_LINUX, and I'll do ARM64_LINUX hopefully soon. It makes the system much much portable, and cuts the dependency on gcc, and LLVM. It enables not using an old gcc. It enables a level of portability of the system approaching helloworld. It increases debugability of the system where m3gdb doesn't run -- Mac and Windows. Can this can be further improved. C and C++ really are very portable. I don't understand why people claim otherwise. There are millions of lines of code that are OS-portable and/or architecture-portable in C and C++. Far more than Modula-3. Yes I'd use shifting and masking and it would be correct and bugfree. I'd use the Word interface. Nothing much is at fault. If fault must be found, it is in the tension between self-written languages is good for understanding and proving the language, and bad for bootstrapping. People struggle to build our system. This would provide a step forward the typical: tar xf cm3.tar.gz. cd cm3 ./configure (not yet) make (already, but only builds cm3) This already works, but needs work to build the entire system, beyond cm3. By removing the endian-ness dependency, the distributed cm3.tar.gz can be one and the same for more systems, i.e. sparc and powerpc and x86. Beyond that, there is word size and NT/Posix dependency. - Jay ________________________________ From: Darko Volaric > Sent: Monday, May 14, 2018 6:30 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] endianness again So what I'm guessing is you're proposing that instead of the compiler figuring out the bit operations for extracting bit fields you're going to do them yourself? That sounds like a terrible idea and likely to introduce bugs that are hard to identify and fix and well as code that is hard to maintain. I also think generating C is a terrible idea. Why are we hacking the compiler to support this misfeature? If it's to support portability then I think it's the wrong approach and not likely to actually improve portability since C is less portable than M3. The current structure of the compiler and libraries is at fault here wrt portability. I don't know about other users, but I use M3 becuase it's not C. I'd like to move the compiler toward having no C sources and no C generation. On Mon, May 14, 2018 at 10:43 AM, Jay K > wrote: I'd like to for the system from m3core up to cm3 to generate the same C for all systems. It almost does. This enable C backend output as a distribution format. In order to get closer to that, I'd like bitfields removed from this part of the system. In particular in RT0.i3. To replace the bitfields with some amount of shifting and masking. Maybe ok? Alternatively..depending on interop needs, can we change the bitfield layout rules to be endian-independent? The problem with that is it hurts C interop, but arguably in an obscure area. Does anyone really use bitfields with <*external*> and depend on it working? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 01:29:23 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 23:29:23 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , , Message-ID: SYSTEM_CC => SYSTEM_CXX? Or use SYSTEM_CC but with -xc++ or /TP. I'll try the second. Well, er.. no. gcc -xc++ 1.cpp compiles as C++ but then gcc 1.o doesn't link nor gcc -xc++ 1.o So I think we need to fork like that, and then use C++ compiler to link, always. ok? Or else say things like -lc++ or -lstdc++ which are more platform specific than I'd like. I seriously would like cm3 to just output CMakeLists.txt or Makefile.am and leave the details of finding and running compilers/linkers to other. And of those two, cmake. Begetting ninja. - Jay ________________________________ From: Hosking, Antony L Sent: Tuesday, May 15, 2018 9:29 PM To: Jay K Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 cxx_source Sent from my iPad On May 16, 2018, at 7:11 AM, Jay K > wrote: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 01:43:44 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 23:43:44 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , , , Message-ID: Ugh. We have .c files. We used to be able to compile as C++ with g++ foo.c. Now that warns. We have to say gcc -xc++ foo.c or g++ -xc++ foo.c. But if you link with gcc -xc++ foo.o -- error. You have to link with g++ foo.o. I'd rather not rename files. So we need a C++ compiler and a C++ linker. They kinda can't be the same. The C linker won't necessarily do. - Jay ________________________________ From: Jay K Sent: Tuesday, May 15, 2018 11:29 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 SYSTEM_CC => SYSTEM_CXX? Or use SYSTEM_CC but with -xc++ or /TP. I'll try the second. Well, er.. no. gcc -xc++ 1.cpp compiles as C++ but then gcc 1.o doesn't link nor gcc -xc++ 1.o So I think we need to fork like that, and then use C++ compiler to link, always. ok? Or else say things like -lc++ or -lstdc++ which are more platform specific than I'd like. I seriously would like cm3 to just output CMakeLists.txt or Makefile.am and leave the details of finding and running compilers/linkers to other. And of those two, cmake. Begetting ninja. - Jay ________________________________ From: Hosking, Antony L Sent: Tuesday, May 15, 2018 9:29 PM To: Jay K Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 cxx_source Sent from my iPad On May 16, 2018, at 7:11 AM, Jay K > wrote: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 01:44:37 2018 From: jayk123 at hotmail.com (Jay K) Date: Tue, 15 May 2018 23:44:37 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , , , , Message-ID: How about CXX and CXXLD. And drop the "SYSTEM_" prefix? - Jay ________________________________ From: Jay K Sent: Tuesday, May 15, 2018 11:43 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Ugh. We have .c files. We used to be able to compile as C++ with g++ foo.c. Now that warns. We have to say gcc -xc++ foo.c or g++ -xc++ foo.c. But if you link with gcc -xc++ foo.o -- error. You have to link with g++ foo.o. I'd rather not rename files. So we need a C++ compiler and a C++ linker. They kinda can't be the same. The C linker won't necessarily do. - Jay ________________________________ From: Jay K Sent: Tuesday, May 15, 2018 11:29 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 SYSTEM_CC => SYSTEM_CXX? Or use SYSTEM_CC but with -xc++ or /TP. I'll try the second. Well, er.. no. gcc -xc++ 1.cpp compiles as C++ but then gcc 1.o doesn't link nor gcc -xc++ 1.o So I think we need to fork like that, and then use C++ compiler to link, always. ok? Or else say things like -lc++ or -lstdc++ which are more platform specific than I'd like. I seriously would like cm3 to just output CMakeLists.txt or Makefile.am and leave the details of finding and running compilers/linkers to other. And of those two, cmake. Begetting ninja. - Jay ________________________________ From: Hosking, Antony L Sent: Tuesday, May 15, 2018 9:29 PM To: Jay K Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 cxx_source Sent from my iPad On May 16, 2018, at 7:11 AM, Jay K > wrote: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 11:03:57 2018 From: jayk123 at hotmail.com (Jay K) Date: Wed, 16 May 2018 09:03:57 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , Message-ID: 3 of those are my understanding and I hadn't considered assembly. g++ for linking is kinda required, if there is any C++ source, maybe depending on what the C++ sources does. gcc -lc++ or gcc-lstdc++ would also work, but you have to figure out which. I'll try to get cxx_source in. But please consider ifdef __cplusplus extern "C". C++ is nearly a proper superset of C. And then test the tree once with c_source, and then switch it all to cxx. Maybe leave a test case for c_source. I'd really like to be able to have tests that are required fail. c_source(foo.c) #ifdef __cplusplus #error Here are some "rules" for converting C to C++: extern "C" fixup/cast between int and enum cast from void* to non-void* -- this is the biggest don't use designated initializers if you are using C99 and declare variables "late", and initialize with declaration, and goto around, then then int foo = 1; to int foo; foo = 1; C++ does not allow goto around initialization, but C99 does. rename keywords like this to self, virtual to virtual_, typeid to type, template to generic, etc. gcc went through this stage of C that is valid C++ and eventually embraced C++ i've pondered this space a long time, tried to be open minded, experimented with various hypotheses ("C is good", "C++ is bad") and always arrive at the same conclusion. C is pretty horrible, C++ is pretty excellent. All the criticisms I hear about C++ I have found to be untrue. It does have flaws but you rarely hear them: no safety (being slightly mitigated), slow builds (being fixed), no "portable executable". These days almost everyone (kids) has moved on to java/c#/javascript/typescript/go/rust so maybe both are almost irrelevant. Systems with optional safety are rare: Modula3 and C#, and the unsafe part of C# is rarely used Safe systems w/o GC are rare: Rust. Safe systems w/o JIT are somewhat rare: Modula3, Rust, Go, .Net Native, optionallly Mono ("AOT"). Though I've heard that due to layout/locality, JIT is actually preferred for performance. Self-hosted systems are somewhat rare: C++, Modula3, go, not sure of the list. (C is no longer self hosted -- the C compilers are written in C++). Lack of module system and attendant slow compilation is rare but two big exsmples: C, C++ (being fixed, slowly) - Jay ________________________________ From: Mika Nystrom Sent: Wednesday, May 16, 2018 8:49 AM To: Jay K Cc: Hosking, Antony L; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 The combination I emailed out was the only that I could get to work: gcc for C source g++ for C++ source gcc for assembly g++ for linking I special-cased _m3main.c I agree cxx_source is the way to do it. Mika On Tue, May 15, 2018 at 4:44 PM, Jay K > wrote: How about CXX and CXXLD. And drop the "SYSTEM_" prefix? - Jay ________________________________ From: Jay K > Sent: Tuesday, May 15, 2018 11:43 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Ugh. We have .c files. We used to be able to compile as C++ with g++ foo.c. Now that warns. We have to say gcc -xc++ foo.c or g++ -xc++ foo.c. But if you link with gcc -xc++ foo.o -- error. You have to link with g++ foo.o. I'd rather not rename files. So we need a C++ compiler and a C++ linker. They kinda can't be the same. The C linker won't necessarily do. - Jay ________________________________ From: Jay K > Sent: Tuesday, May 15, 2018 11:29 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 SYSTEM_CC => SYSTEM_CXX? Or use SYSTEM_CC but with -xc++ or /TP. I'll try the second. Well, er.. no. gcc -xc++ 1.cpp compiles as C++ but then gcc 1.o doesn't link nor gcc -xc++ 1.o So I think we need to fork like that, and then use C++ compiler to link, always. ok? Or else say things like -lc++ or -lstdc++ which are more platform specific than I'd like. I seriously would like cm3 to just output CMakeLists.txt or Makefile.am and leave the details of finding and running compilers/linkers to other. And of those two, cmake. Begetting ninja. - Jay ________________________________ From: Hosking, Antony L > Sent: Tuesday, May 15, 2018 9:29 PM To: Jay K Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 cxx_source Sent from my iPad On May 16, 2018, at 7:11 AM, Jay K > wrote: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed May 16 14:33:24 2018 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2018 22:33:24 +1000 Subject: [M3devel] LongrealType and RealType (again) In-Reply-To: References: Message-ID: <417C82BB-8DD6-49D4-BDBD-C64ABACE92CF@cs.purdue.edu> I?m struggling to remember why?it has been a long time. I?m sure I had a good reason. > On May 15, 2018, at 7:50 PM, Mika Nystrom wrote: > > actually never mind this. > > I'll do the one-time update of all the code I have, it's not such a big deal after all. > > LongrealType -> LongReal > > RealType -> Real > > (here and there) > > On Tue, May 15, 2018 at 1:17 AM, Mika Nystrom > wrote: > Can someone explain this to me (again): > > commit 0452245d9e1866f4290b9ba4ea224f89e5ed812f > Author: Anthony Hosking > > Date: Wed Dec 23 21:43:37 2009 +0000 > > Move libm3/src/types into m3core. > Note that LongrealType and RealType have been folded into m3core/src/float. > Clients will need adjustment. > > ---- > > LongrealType and RealType are needed to instantiate generics...why were they removed from libm3? Does it cause trouble to have them there? > > I am not sure it makes sense to go back and rewrite everything. And have conditional compilation (UGH!!!!) depending on the version of CM3. > > I ran into this before, years ago... what did we say? > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 20:56:11 2018 From: jayk123 at hotmail.com (Jay K) Date: Wed, 16 May 2018 18:56:11 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , Message-ID: Are you using assembly like "gcc -S" to import arbitrary language? Interesting. I guess the quake-extensibility might not suffice, what with all the file name extension logic in Mod3? But I guess that was also deliberate, for performance, 20 years ago. While I still like C++ I do wonder if the efficiency equation has shifted enough, everyone using "slow" languages. i.e. we could just move the logic back to quake, restore extensibility. Or maybe just hooks. Or maybe it is already adequate. As I understand, we used to have a lot more quake and it was implemented more slowly. Critical Mass rewrote a bunch in Modula-3, and wrote a compile+interpret-bytecode for Quake. Anyway, assembly can/should work. Hopefully we don't have much, except all the m3cc output, hopefully to be replaced by C/C++. ? - Jay ________________________________ From: Mika Nystrom Sent: Wednesday, May 16, 2018 9:09 AM To: Jay K Cc: Hosking, Antony L; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Yet another dimension you can consider is JVM or not. Anyhow, the C code that I deal with is imported from elsewhere. It may change from time to time. Ideally I don't want to go through and edit it. And yeah there's the assembly. I have quite a lot of it because of ifort---don't really want to integrate Fortran in cm3 but I guess that would be the Right Way To Do It.... On Wed, May 16, 2018 at 2:03 AM, Jay K > wrote: 3 of those are my understanding and I hadn't considered assembly. g++ for linking is kinda required, if there is any C++ source, maybe depending on what the C++ sources does. gcc -lc++ or gcc-lstdc++ would also work, but you have to figure out which. I'll try to get cxx_source in. But please consider ifdef __cplusplus extern "C". C++ is nearly a proper superset of C. And then test the tree once with c_source, and then switch it all to cxx. Maybe leave a test case for c_source. I'd really like to be able to have tests that are required fail. c_source(foo.c) #ifdef __cplusplus #error Here are some "rules" for converting C to C++: extern "C" fixup/cast between int and enum cast from void* to non-void* -- this is the biggest don't use designated initializers if you are using C99 and declare variables "late", and initialize with declaration, and goto around, then then int foo = 1; to int foo; foo = 1; C++ does not allow goto around initialization, but C99 does. rename keywords like this to self, virtual to virtual_, typeid to type, template to generic, etc. gcc went through this stage of C that is valid C++ and eventually embraced C++ i've pondered this space a long time, tried to be open minded, experimented with various hypotheses ("C is good", "C++ is bad") and always arrive at the same conclusion. C is pretty horrible, C++ is pretty excellent. All the criticisms I hear about C++ I have found to be untrue. It does have flaws but you rarely hear them: no safety (being slightly mitigated), slow builds (being fixed), no "portable executable". These days almost everyone (kids) has moved on to java/c#/javascript/typescript/go/rust so maybe both are almost irrelevant. Systems with optional safety are rare: Modula3 and C#, and the unsafe part of C# is rarely used Safe systems w/o GC are rare: Rust. Safe systems w/o JIT are somewhat rare: Modula3, Rust, Go, .Net Native, optionallly Mono ("AOT"). Though I've heard that due to layout/locality, JIT is actually preferred for performance. Self-hosted systems are somewhat rare: C++, Modula3, go, not sure of the list. (C is no longer self hosted -- the C compilers are written in C++). Lack of module system and attendant slow compilation is rare but two big exsmples: C, C++ (being fixed, slowly) - Jay ________________________________ From: Mika Nystrom > Sent: Wednesday, May 16, 2018 8:49 AM To: Jay K Cc: Hosking, Antony L; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 The combination I emailed out was the only that I could get to work: gcc for C source g++ for C++ source gcc for assembly g++ for linking I special-cased _m3main.c I agree cxx_source is the way to do it. Mika On Tue, May 15, 2018 at 4:44 PM, Jay K > wrote: How about CXX and CXXLD. And drop the "SYSTEM_" prefix? - Jay ________________________________ From: Jay K > Sent: Tuesday, May 15, 2018 11:43 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Ugh. We have .c files. We used to be able to compile as C++ with g++ foo.c. Now that warns. We have to say gcc -xc++ foo.c or g++ -xc++ foo.c. But if you link with gcc -xc++ foo.o -- error. You have to link with g++ foo.o. I'd rather not rename files. So we need a C++ compiler and a C++ linker. They kinda can't be the same. The C linker won't necessarily do. - Jay ________________________________ From: Jay K > Sent: Tuesday, May 15, 2018 11:29 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 SYSTEM_CC => SYSTEM_CXX? Or use SYSTEM_CC but with -xc++ or /TP. I'll try the second. Well, er.. no. gcc -xc++ 1.cpp compiles as C++ but then gcc 1.o doesn't link nor gcc -xc++ 1.o So I think we need to fork like that, and then use C++ compiler to link, always. ok? Or else say things like -lc++ or -lstdc++ which are more platform specific than I'd like. I seriously would like cm3 to just output CMakeLists.txt or Makefile.am and leave the details of finding and running compilers/linkers to other. And of those two, cmake. Begetting ninja. - Jay ________________________________ From: Hosking, Antony L > Sent: Tuesday, May 15, 2018 9:29 PM To: Jay K Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 cxx_source Sent from my iPad On May 16, 2018, at 7:11 AM, Jay K > wrote: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed May 16 23:51:37 2018 From: jayk123 at hotmail.com (Jay K) Date: Wed, 16 May 2018 21:51:37 +0000 Subject: [M3devel] Using g++ to compile C code under cm3 In-Reply-To: References: , Message-ID: > Including the fact that ifort may only run on Linux but its output works on FreeBSD! Indeed, the Unix ABIs are often the same. - Jay ________________________________ From: Mika Nystrom Sent: Wednesday, May 16, 2018 9:35 PM To: Jay K Cc: Hosking, Antony L; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Well, I only do it for Fortran.... it avoids a whole huge set of problems for that particular case. Including the fact that ifort may only run on Linux but its output works on FreeBSD! Another place I might want to do assembly is to do a custom alloca-like thing to do cactus stacks (soon.. I hope...) Otherwise I never see a need for assembly as source. Mika On Wed, May 16, 2018 at 11:56 AM, Jay K > wrote: Are you using assembly like "gcc -S" to import arbitrary language? Interesting. I guess the quake-extensibility might not suffice, what with all the file name extension logic in Mod3? But I guess that was also deliberate, for performance, 20 years ago. While I still like C++ I do wonder if the efficiency equation has shifted enough, everyone using "slow" languages. i.e. we could just move the logic back to quake, restore extensibility. Or maybe just hooks. Or maybe it is already adequate. As I understand, we used to have a lot more quake and it was implemented more slowly. Critical Mass rewrote a bunch in Modula-3, and wrote a compile+interpret-bytecode for Quake. Anyway, assembly can/should work. Hopefully we don't have much, except all the m3cc output, hopefully to be replaced by C/C++. ? - Jay ________________________________ From: Mika Nystrom > Sent: Wednesday, May 16, 2018 9:09 AM To: Jay K Cc: Hosking, Antony L; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Yet another dimension you can consider is JVM or not. Anyhow, the C code that I deal with is imported from elsewhere. It may change from time to time. Ideally I don't want to go through and edit it. And yeah there's the assembly. I have quite a lot of it because of ifort---don't really want to integrate Fortran in cm3 but I guess that would be the Right Way To Do It.... On Wed, May 16, 2018 at 2:03 AM, Jay K > wrote: 3 of those are my understanding and I hadn't considered assembly. g++ for linking is kinda required, if there is any C++ source, maybe depending on what the C++ sources does. gcc -lc++ or gcc-lstdc++ would also work, but you have to figure out which. I'll try to get cxx_source in. But please consider ifdef __cplusplus extern "C". C++ is nearly a proper superset of C. And then test the tree once with c_source, and then switch it all to cxx. Maybe leave a test case for c_source. I'd really like to be able to have tests that are required fail. c_source(foo.c) #ifdef __cplusplus #error Here are some "rules" for converting C to C++: extern "C" fixup/cast between int and enum cast from void* to non-void* -- this is the biggest don't use designated initializers if you are using C99 and declare variables "late", and initialize with declaration, and goto around, then then int foo = 1; to int foo; foo = 1; C++ does not allow goto around initialization, but C99 does. rename keywords like this to self, virtual to virtual_, typeid to type, template to generic, etc. gcc went through this stage of C that is valid C++ and eventually embraced C++ i've pondered this space a long time, tried to be open minded, experimented with various hypotheses ("C is good", "C++ is bad") and always arrive at the same conclusion. C is pretty horrible, C++ is pretty excellent. All the criticisms I hear about C++ I have found to be untrue. It does have flaws but you rarely hear them: no safety (being slightly mitigated), slow builds (being fixed), no "portable executable". These days almost everyone (kids) has moved on to java/c#/javascript/typescript/go/rust so maybe both are almost irrelevant. Systems with optional safety are rare: Modula3 and C#, and the unsafe part of C# is rarely used Safe systems w/o GC are rare: Rust. Safe systems w/o JIT are somewhat rare: Modula3, Rust, Go, .Net Native, optionallly Mono ("AOT"). Though I've heard that due to layout/locality, JIT is actually preferred for performance. Self-hosted systems are somewhat rare: C++, Modula3, go, not sure of the list. (C is no longer self hosted -- the C compilers are written in C++). Lack of module system and attendant slow compilation is rare but two big exsmples: C, C++ (being fixed, slowly) - Jay ________________________________ From: Mika Nystrom > Sent: Wednesday, May 16, 2018 8:49 AM To: Jay K Cc: Hosking, Antony L; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 The combination I emailed out was the only that I could get to work: gcc for C source g++ for C++ source gcc for assembly g++ for linking I special-cased _m3main.c I agree cxx_source is the way to do it. Mika On Tue, May 15, 2018 at 4:44 PM, Jay K > wrote: How about CXX and CXXLD. And drop the "SYSTEM_" prefix? - Jay ________________________________ From: Jay K > Sent: Tuesday, May 15, 2018 11:43 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 Ugh. We have .c files. We used to be able to compile as C++ with g++ foo.c. Now that warns. We have to say gcc -xc++ foo.c or g++ -xc++ foo.c. But if you link with gcc -xc++ foo.o -- error. You have to link with g++ foo.o. I'd rather not rename files. So we need a C++ compiler and a C++ linker. They kinda can't be the same. The C linker won't necessarily do. - Jay ________________________________ From: Jay K > Sent: Tuesday, May 15, 2018 11:29 PM To: Hosking, Antony L Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 SYSTEM_CC => SYSTEM_CXX? Or use SYSTEM_CC but with -xc++ or /TP. I'll try the second. Well, er.. no. gcc -xc++ 1.cpp compiles as C++ but then gcc 1.o doesn't link nor gcc -xc++ 1.o So I think we need to fork like that, and then use C++ compiler to link, always. ok? Or else say things like -lc++ or -lstdc++ which are more platform specific than I'd like. I seriously would like cm3 to just output CMakeLists.txt or Makefile.am and leave the details of finding and running compilers/linkers to other. And of those two, cmake. Begetting ninja. - Jay ________________________________ From: Hosking, Antony L > Sent: Tuesday, May 15, 2018 9:29 PM To: Jay K Cc: Mika Nystrom; m3devel at elegosoft.com; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Re: Using g++ to compile C code under cm3 cxx_source Sent from my iPad On May 16, 2018, at 7:11 AM, Jay K > wrote: It was me long ago. All the code in cm3 should be valid C++98 and C89. But not K&R C. Imho all C code should be valid C++. I've made it so a few times. I agree it isn't automatic though. I guess a new directive is in order -- cplusplus_source? And change the entire system to use it. Can we use + signs? :) c++_source()? - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, May 15, 2018 8:41 AM To: m3devel at elegosoft.com; Jay K; jay.krell at cornell.edu; Antony Hosking; rodney_bates at lcwb.coop; Peter Mckinna Subject: Using g++ to compile C code under cm3 So someone has changed the CM3 system to use g++ rather than a C compiler to compile files that are called out as "c_source". This can't be right. c_source is written in C, not C++. It's breaking tons and tons and tons of my old code, which has never had problems before. If you want to use C++ code, can we please call it cxx_source or similar, and use a C compiler to compile C code? here's an example: sccj004186> cat CTZ.i3 (* $Id: CTZ.i3,v 1.3 2011/02/10 00:16:47 mika Exp $ *) INTERFACE CTZ; FROM Ctypes IMPORT const_char_star, char_star; <*EXTERNAL setenv*> PROCEDURE setenv(name, value : const_char_star; overwrite : INTEGER) : INTEGER; <*EXTERNAL getenv*> PROCEDURE getenv(name : const_char_star) : char_star; <*EXTERNAL CTZ__setTZ *> PROCEDURE setTZ(name : const_char_star); <*OBSOLETE*> <*EXTERNAL CTZ__setenv_TZ_America_New_York *> PROCEDURE setenv_TZ_America_New_York(); END CTZ. sccj004186> cat CTZ.c /* $Id: CTZ.c,v 1.1 2011/01/01 20:16:29 mika Exp $ */ #include #include #include void CTZ__setenv_TZ_America_New_York(void) { int res = setenv("TZ","America/New_York",1); assert(res==0); } void CTZ__setTZ(const char *tzname) { int res = setenv("TZ",tzname,1); assert(res==0); tzset(); } sccj004186> nm ../AMD64_LINUX/libcit_util.a | grep CTZ CTZ_i.o: 0000000000000000 T CTZ_I3 0000000000000000 d MI_CTZ CTZ.o: 000000000000004b T _Z10CTZ__setTZPKc 0000000000000000 T _Z31CTZ__setenv_TZ_America_New_Yorkv 0000000000000070 r _ZZ10CTZ__setTZPKcE19__PRETTY_FUNCTION__ 0000000000000040 r _ZZ31CTZ__setenv_TZ_America_New_YorkvE19__PRETTY_FUNCTION__ U CTZ_I3 U CTZ__setTZ I expected that void CTZ__setTZ would match <*EXTERNAL CTZ__setTZ*> but the C++ compiler mangles the name into ... some nonsense. Yes I realize I can do some sort of C linkage annotation. But I already called out my source as being in C! sccj004186> grep CTZ !$ grep CTZ m3makefile interface ("CTZ") c_source ("CTZ") Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri May 18 09:44:34 2018 From: jayk123 at hotmail.com (Jay K) Date: Fri, 18 May 2018 07:44:34 +0000 Subject: [M3devel] dual license? Message-ID: Can we dual license "our" stuff? I mean, I don't want: https://github.com/jaykrell/cm3/tree/master/m3-sys/m3cc/gcc/gcc/m3cg to be locked up in GPL. https://github.com/jaykrell/cm3/blob/master/m3-sys/m3cc/gcc/gcc/m3cg/m3-def.h in particular is fully my doing. parse.c I'd also like to be BSD licensed but I didn't write as much of it. In particular, to write another IR reader or interpreter in C++. https://github.com/jaykrell/cm3/blob/master/m3-sys/m3cc/gcc/gcc/m3cg/m3-def.h is super useful. As the author, I just can do it, right? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Fri May 18 09:58:41 2018 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 18 May 2018 09:58:41 +0200 Subject: [M3devel] dual license? In-Reply-To: References: Message-ID: <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> On Fri, 18 May 2018 07:44:34 +0000 Jay K wrote: > Can we dual license "our" stuff? > > I mean, I don't want: > > > https://github.com/jaykrell/cm3/tree/master/m3-sys/m3cc/gcc/gcc/m3cg > > > to be locked up in GPL. > > https://github.com/jaykrell/cm3/blob/master/m3-sys/m3cc/gcc/gcc/m3cg/m3-def.h > > in particular is fully my doing. parse.c I'd also like to be BSD licensed but I didn't write as much of it. > > In particular, to write another IR reader or interpreter in C++. > https://github.com/jaykrell/cm3/blob/master/m3-sys/m3cc/gcc/gcc/m3cg/m3-def.h is super useful. > > As the author, I just can do it, right? I don't think you can change the license of m3cg. GPL is viral. Please be careful there or we will get sued ;-) Anything that links with GPL code is infected. Some years ago I tried to contact Farshad Nayeri from Critical Mass about the possibility of a license change, but never got an answer. This would only cover the CM3 license of course, not cm3cg. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jayk123 at hotmail.com Fri May 18 21:07:00 2018 From: jayk123 at hotmail.com (Jay K) Date: Fri, 18 May 2018 19:07:00 +0000 Subject: [M3devel] dual license? In-Reply-To: <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> References: , <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> Message-ID: So. I don't fully understand this idea of "dual licensing", but it exists. For example Qt. And the owner of the code gets a say, can change his mind somehow. But the code has been linked to gcc, so I don't know if that takes it over or what. - Jay ________________________________ From: Olaf Wagner Sent: Friday, May 18, 2018 7:58 AM To: Jay K Cc: m3devel Subject: Re: [M3devel] dual license? On Fri, 18 May 2018 07:44:34 +0000 Jay K wrote: > Can we dual license "our" stuff? > > I mean, I don't want: > > > https://github.com/jaykrell/cm3/tree/master/m3-sys/m3cc/gcc/gcc/m3cg > > > to be locked up in GPL. > > https://github.com/jaykrell/cm3/blob/master/m3-sys/m3cc/gcc/gcc/m3cg/m3-def.h > > in particular is fully my doing. parse.c I'd also like to be BSD licensed but I didn't write as much of it. > > In particular, to write another IR reader or interpreter in C++. > https://github.com/jaykrell/cm3/blob/master/m3-sys/m3cc/gcc/gcc/m3cg/m3-def.h is super useful. > > As the author, I just can do it, right? I don't think you can change the license of m3cg. GPL is viral. Please be careful there or we will get sued ;-) Anything that links with GPL code is infected. Some years ago I tried to contact Farshad Nayeri from Critical Mass about the possibility of a license change, but never got an answer. This would only cover the CM3 license of course, not cm3cg. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun May 20 02:44:15 2018 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 19 May 2018 20:44:15 -0400 Subject: [M3devel] dual license? In-Reply-To: References: <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> Message-ID: <20180520004415.GB18272@topoi.pooq.com> On Fri, May 18, 2018 at 07:07:00PM +0000, Jay K wrote: > So. I don't fully understand this idea of "dual licensing", but it exists. For example Qt. > > And the owner of the code gets a say, can change his mind somehow. > > But the code has been linked to gcc, so I don't know if that takes it over or what. If you link with gpl code, and distribute the binary, you have to distribute the source code under GPL and make it available to everyone you give the binary to. You can still distribute your code under as many extra licences as you want. You can evern give different licenses to different people if you want, but giving linked binaries containing other code that's GPL's binds you to release your source GPL's too. For dual licencing, you usually give the recipient a choice of licences. Saying GPL2+ or MIT or SD or SRC is perfectly valid. It the recipient then links your code with non-GPL-compliant code, that's OK, as long as one of the other licences allows it. Dual licencing makes it more permissive, not less. -- hendrik From lists at darko.org Sun May 20 02:51:33 2018 From: lists at darko.org (Darko Volaric) Date: Sun, 20 May 2018 02:51:33 +0200 Subject: [M3devel] dual license? In-Reply-To: <20180520004415.GB18272@topoi.pooq.com> References: <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> <20180520004415.GB18272@topoi.pooq.com> Message-ID: Yes, but it's all academic if said code is derived from GPL code or depends on GPL code since the recipient of the more permissive license is forced into GPL licensing anyway, as is the case with cm3cg. > On May 20, 2018, at 2:44 AM, Hendrik Boom wrote: > > On Fri, May 18, 2018 at 07:07:00PM +0000, Jay K wrote: >> So. I don't fully understand this idea of "dual licensing", but it exists. For example Qt. >> >> And the owner of the code gets a say, can change his mind somehow. >> >> But the code has been linked to gcc, so I don't know if that takes it over or what. > > If you link with gpl code, and distribute the binary, you have to > distribute the source code under GPL and make it available to everyone > you give the binary to. > > You can still distribute your code under as many extra licences as you want. > > You can evern give different licenses to different people if you want, > but giving linked binaries containing other code that's GPL's binds > you to release your source GPL's too. > > For dual licencing, you usually give the recipient a choice of > licences. Saying GPL2+ or MIT or SD or SRC is perfectly valid. > > It the recipient then links your code with non-GPL-compliant code, > that's OK, as long as one of the other licences allows it. > > Dual licencing makes it more permissive, not less. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Sun May 20 13:24:57 2018 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 20 May 2018 07:24:57 -0400 Subject: [M3devel] dual license? In-Reply-To: References: <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> <20180520004415.GB18272@topoi.pooq.com> Message-ID: <20180520112457.GB29645@topoi.pooq.com> On Sun, May 20, 2018 at 02:51:33AM +0200, Darko Volaric wrote: > Yes, but it's all academic if said code is derived from GPL code or depends on GPL code since the recipient of the more permissive license is forced into GPL licensing anyway, as is the case with cm3cg. In the short term, yes, it is academic. In the long term, perhaps not. Someone might replace the GPL code with other, sufficiently compatible code. Then the availability of other licences becomes important. So dual-licence your code, if you object to GPL but want to remain compatible with others' GPL code. In the case of Modula 3, the licence we have to struggle with is the noncontagious SRC licence. Licencing new code multiply under SRC, GPL, and MIT will open the maximum in potential licence compatibility in possibly the far future. -- hendrik > > > > On May 20, 2018, at 2:44 AM, Hendrik Boom wrote: > > > > On Fri, May 18, 2018 at 07:07:00PM +0000, Jay K wrote: > >> So. I don't fully understand this idea of "dual licensing", but it exists. For example Qt. > >> > >> And the owner of the code gets a say, can change his mind somehow. > >> > >> But the code has been linked to gcc, so I don't know if that takes it over or what. > > > > If you link with gpl code, and distribute the binary, you have to > > distribute the source code under GPL and make it available to everyone > > you give the binary to. > > > > You can still distribute your code under as many extra licences as you want. > > > > You can evern give different licenses to different people if you want, > > but giving linked binaries containing other code that's GPL's binds > > you to release your source GPL's too. > > > > For dual licencing, you usually give the recipient a choice of > > licences. Saying GPL2+ or MIT or SD or SRC is perfectly valid. > > > > It the recipient then links your code with non-GPL-compliant code, > > that's OK, as long as one of the other licences allows it. > > > > Dual licencing makes it more permissive, not less. > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From jayk123 at hotmail.com Sun May 20 23:23:10 2018 From: jayk123 at hotmail.com (Jay K) Date: Sun, 20 May 2018 21:23:10 +0000 Subject: [M3devel] slisp.Copy infinite recursion? Message-ID: == package /s/cm3/m3-libs/slisp == +++ /cm3.amd64/bin/cm3 -keep -DM3_BACKEND_MODE=C -build -DROOT=/s/cm3 +++ --- building in AMD64_DARWINc --- ignoring ../src/m3overrides new source -> compiling SLisp.m3 g++ -g -fPIC -Wall -Werror -Wno-return-type -Wno-missing-braces -m64 -arch x86_64 -x c++ -funwind-tables -g -xc++ -c SLisp.m3.c -o SLisp_m.o ../src/SLisp.m3:1131:1: error: all paths through this function will call itself [-Werror,-Winfinite-recursion] { PROCEDURE Copy (x: RefList.T): RefList.T = BEGIN RETURN RefList.Cons (x.head, Copy (x.tail)); END Copy; ? The C backend is correct and this code is just broken, right? I mean, it isn't like there are any kind of "lazy" semantics, or the order of evaluation can help, right? Maybe inlining could fix it, maybe. But it'd be cross module, which none of the backends do. The cvswww thing doesn't seem available any longer. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon May 21 03:53:47 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 21 May 2018 01:53:47 +0000 Subject: [M3devel] dual license? In-Reply-To: References: <20180518095841.00ffed3be243ecf8b3f4a943@elegosoft.com> <20180520004415.GB18272@topoi.pooq.com>, Message-ID: On useful file in particular, does not depend on or derive from GPL code. m3-def.h and m3cg.h The integer/token/id parsing would be nice too. i.e. to write another backend in C++ that reads the persistent IR. Or maybe a function-call based one. - Jay ________________________________ From: M3devel on behalf of Darko Volaric Sent: Sunday, May 20, 2018 12:51 AM To: Hendrik Boom Cc: m3devel at elegosoft.com Subject: Re: [M3devel] dual license? Yes, but it's all academic if said code is derived from GPL code or depends on GPL code since the recipient of the more permissive license is forced into GPL licensing anyway, as is the case with cm3cg. > On May 20, 2018, at 2:44 AM, Hendrik Boom wrote: > > On Fri, May 18, 2018 at 07:07:00PM +0000, Jay K wrote: >> So. I don't fully understand this idea of "dual licensing", but it exists. For example Qt. >> >> And the owner of the code gets a say, can change his mind somehow. >> >> But the code has been linked to gcc, so I don't know if that takes it over or what. > > If you link with gpl code, and distribute the binary, you have to > distribute the source code under GPL and make it available to everyone > you give the binary to. > > You can still distribute your code under as many extra licences as you want. > > You can evern give different licenses to different people if you want, > but giving linked binaries containing other code that's GPL's binds > you to release your source GPL's too. > > For dual licencing, you usually give the recipient a choice of > licences. Saying GPL2+ or MIT or SD or SRC is perfectly valid. > > It the recipient then links your code with non-GPL-compliant code, > that's OK, as long as one of the other licences allows it. > > Dual licencing makes it more permissive, not less. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon May 21 07:35:06 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 21 May 2018 05:35:06 +0000 Subject: [M3devel] slisp.Copy infinite recursion? In-Reply-To: References: , Message-ID: I commited a fix to master. https://github.com/modula3/cm3/commits/master Protest if we should not self-merge our own PRs to master. - Jay ________________________________ From: Hosking, Antony L Sent: Monday, May 21, 2018 2:06 AM To: Jay K Cc: m3devel Subject: Re: [M3devel] slisp.Copy infinite recursion? Yes, that is broken source code from the looks. No test for the NULL list. On May 21, 2018, at 7:23 AM, Jay K > wrote: == package /s/cm3/m3-libs/slisp == +++ /cm3.amd64/bin/cm3 -keep -DM3_BACKEND_MODE=C -build -DROOT=/s/cm3 +++ --- building in AMD64_DARWINc --- ignoring ../src/m3overrides new source -> compiling SLisp.m3 g++ -g -fPIC -Wall -Werror -Wno-return-type -Wno-missing-braces -m64 -arch x86_64 -x c++ -funwind-tables -g -xc++ -c SLisp.m3.c -o SLisp_m.o ../src/SLisp.m3:1131:1: error: all paths through this function will call itself [-Werror,-Winfinite-recursion] { PROCEDURE Copy (x: RefList.T): RefList.T = BEGIN RETURN RefList.Cons (x.head, Copy (x.tail)); END Copy; ? The C backend is correct and this code is just broken, right? I mean, it isn't like there are any kind of "lazy" semantics, or the order of evaluation can help, right? Maybe inlining could fix it, maybe. But it'd be cross module, which none of the backends do. The cvswww thing doesn't seem available any longer. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon May 21 07:40:09 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 21 May 2018 05:40:09 +0000 Subject: [M3devel] grisu and C backend Message-ID: grisu gives C backend some grief I'll investigate.. -> linking shobjcodegen g++ -g -fPIC -Wall -Werror -Wno-return-type -Wno-missing-braces -m64 -arch x86_64 -x c++ -funwind-tables -g -xc++ -c _m3main.c -o _m3main.o *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/float/Common/grisu/IEEE.m3", line 155 *** -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon May 21 07:42:50 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 21 May 2018 05:42:50 +0000 Subject: [M3devel] longint in m3core? Message-ID: Can m3core use longint now? I'm on the fence. I suppose, even with bootstrapping from old, m3core can depend on newest. i.e. it is only libraries above libm3 that are built with old compiler It is in use in grisu. Even higher level stuff maybe ok -- we could remove the multi-precision stuff perhaps, perhaps.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon May 21 08:11:07 2018 From: jayk123 at hotmail.com (Jay K) Date: Mon, 21 May 2018 06:11:07 +0000 Subject: [M3devel] longint in m3core? In-Reply-To: <6ABC9826-71AA-4EDC-8CC6-A9371D1DCE8C@purdue.edu> References: , <6ABC9826-71AA-4EDC-8CC6-A9371D1DCE8C@purdue.edu> Message-ID: It is in use already, not by me. Can you be specific wrt bugs? I know NT386 lacked it a while, maybe into the previous (2010) release but it's been there a while. But that doesn't matter. Old releases do not have to build current m3core. - Jay ________________________________ From: Hosking, Antony L Sent: Monday, May 21, 2018 5:49 AM To: Jay K Cc: m3devel Subject: Re: [M3devel] longint in m3core? I would avoid for now. I believe there are some outstanding compiler bugs that have not yet been addressed... On May 21, 2018, at 3:42 PM, Jay K > wrote: Can m3core use longint now? I'm on the fence. I suppose, even with bootstrapping from old, m3core can depend on newest. i.e. it is only libraries above libm3 that are built with old compiler It is in use in grisu. Even higher level stuff maybe ok -- we could remove the multi-precision stuff perhaps, perhaps.. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: