From antony.hosking at anu.edu.au  Tue Jan  4 05:38:32 2022
From: antony.hosking at anu.edu.au (Tony Hosking)
Date: Tue, 4 Jan 2022 04:38:32 +0000
Subject: [M3devel] significant cm3 bootstrap/wiki/docs progress
In-Reply-To: <086ae65e-d0c5-4684-93c2-842c19300409@www.fastmail.com>
References: <modula3/cm3/issues/13@github.com>
 <modula3/cm3/issues/13/999590388@github.com>
 <MWHPR1401MB1951EE6CCD9CA958FF412117E67D9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <95addb8d-4246-4768-a6fd-1707bda5818f@www.fastmail.com>
 <MWHPR1401MB1951CB4F922E9D140CE3DB94E67F9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <086ae65e-d0c5-4684-93c2-842c19300409@www.fastmail.com>
Message-ID: <SYBP282MB27550E758AD7A467C3BA9628B24A9@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>

Hi Eric,

If you?re interested in putting some love into the GC end of things and have questions let me know. I have broad experience in GC and also specifically with the current CM3.

Cheers,
Tony

From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Eric Sessoms <coder5506 at pobox.com>
Date: Saturday, December 25, 2021 at 3:19 AM
To: Jay K <jayk123 at hotmail.com>, m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] significant cm3 bootstrap/wiki/docs progress
On Fri, Dec 24, 2021, at 5:41 AM, Jay K wrote:
> I can probably get you OSF/1 but I hesitate to..just sending this
> mail...your time is better
> spent in the direction you are already going. Seriously, at some point,
> say no.
> Requests will only get more ridiculous until you do. :)

That's a fair point.  Everybody has their own priorities.  My priority is using cm3 at work, which means I needed first and foremost a stable and reliable compiler (CI/CD) and support for the platforms I use (i.e., fixing macOS), and both of those together led to working on builds and installs.

I recognize a certain responsibility goes along with that.  Nobody wants yet another build system, so if I'm going to do it, I need to do it right and not just make the situation worse.  So yeah, that means making sure it works for as many platforms as I reasonably can.

Now that the initial hurdles are over I do want to spend more time using cm3 and less time working on cm3, but I don't plan to leave a job half-done.

> I don't know if you have seen the whole cminstall thing.

Well, it assumes an existing Modula-3 compiler or a binary distribution, so it wasn't really suitable.

> You know, the stuff about "finding" x11, odbc, etc.
> these can?should be handled by autoconf or cmake or pkg-config.

It can and should.  I spent some time writing autoconf-style functionality in quake (finding a few quake bugs along the way) before discovering how fundamentally limited the m3build stuff is.  It is possible to greatly improve the current m3makefiles, but I think the effort is better spent to just obsolete them.

> When quake was written, cmake didn't exist, and autoconf

Yeah, these things all make sense in context.  I'm sure it was the right move at the time.  And it is easy to use, but is is limiting.  Autoconf and cmake can be just as easy for end users or new contributors... once it's setup.

> Do realize that cm3 bootstrap itself is available
> as one .cpp file...however...I don't know. Maybe
> that was a bad idea. It is unwieldy when it has errors

I thought the single-file approach was great.  I know you said some systems couldn't handle it, but I never had any problems.  What finally decided me in favor of multiple files was the need to include mklib.exe for windows.  So now I need to build two EXEs, both statically linked, sharing 90% of the same code.  Individual files make for a smaller distribution.

> Years ago the OpenBSD position included that gzip is
> fast and low memory on all systems and they insisted on using it,
> vs. higher compression ratios.

Fair.  But where the newer algorithms really suffer is in the compressing, less so the decompressing.  I just did a quick test:  our 40MB xz decompressed in 2.5s and the equivalent 70MB gz decompressed in 1.5s.  I feel like you'll lose that second downloading the extra 30MB.

> In your copious time on this project you may want to "restore"
> building of .msis. That is another kinda dent in this battle.

Agree.  MSIs, DEBs, etc., all need to come back.  But first will be Gentoo ebuilds b/c that's what I personally use.  :)

> vs. both. One might argue that Windows users are fine with binaries,
> but this is a gross generalization that mostly confuses axes.

I did consider that, but you can't realistically use cm3 on Windows without a supporting compiler, so it's a hurdle the user will need to overcome at some point.  MSIs are convenient, but not a top priority.

> though not far and away best. Compiling to LLVM IR is a good choice too,
> and we could even distribute portable textual LLVM IR.

LLVM is also mostly talk these days.  And I'm not picking on our LLVM backend.  They support *way* fewer targets that we do https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclang.llvm.org%2Fdocs%2FUsersManual.html%23target-specific-features-and-limitations&amp;data=04%7C01%7Cantony.hosking%40anu.edu.au%7C20f46e41a2f7475911b108d9c6f90247%7Ce37d725cab5c46249ae5f0533e486437%7C0%7C0%7C637759595952352604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=9bI7PgfHSmw9LvCIskHGDMDNI%2FUBGykvQUoszH2%2FXM0%3D&amp;reserved=0 and their supported host platforms (for distributing IR) are even more limited https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fllvm.org%2Fdocs%2FGettingStarted.html%23requirements&amp;data=04%7C01%7Cantony.hosking%40anu.edu.au%7C20f46e41a2f7475911b108d9c6f90247%7Ce37d725cab5c46249ae5f0533e486437%7C0%7C0%7C637759595952352604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=Q0MBFoZu3ozr8NHsBw8VK4sndGVjLjkuMbDXj4Jffzg%3D&amp;reserved=0

In practical terms, LLVM is just a convenient way to compile to either x86 or Arm, with optimizations that may or may not be worth their cost in added compile time.  It's got some advantages, but I can't see using it for bootstrap.

> While I am airing all of my desires, the garbage collection in cm3 I
> think needs work.

That's something I've been wanting to work on since Day 1!  Before I got sidetracked with the whole build and install thing.  I know the GC has flaws, but it's not Boehm. That's a huge win in my book, since just about everything else not backed by a trillion-dollar corporation uses Boehm.  Boehm is great, but GC is all about tradeoffs, and the tradeoffs it makes are not appropriate for the types of applications I write.

I can't wait to get all this janitorial work out of the way so I can concentrate on the GC.

> I think your cmake work subsumes a lot of this, but you "only"
> did it for bootstrap.

I did.  I could compile the entire system to C and distribute that, but (a) it would be even bigger and (b) it's still a source distribution, so there's no clear advantage.

I can't really take cmake or autotools further until I can get away from quake and its limitations.  Adding for anyone who didn't see the other thread: I don't plan to get rid of quake--I'm sure it will be around forever--I'm just going to add a way to *not* use it.

> Also btw we should be able to produce one C++ form for 32bit and 64bit.
> I was working on it but didn't finish.
(snip)

Thanks for these notes, they will be very useful.
_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&amp;data=04%7C01%7Cantony.hosking%40anu.edu.au%7C20f46e41a2f7475911b108d9c6f90247%7Ce37d725cab5c46249ae5f0533e486437%7C0%7C0%7C637759595952352604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=Lp78o6JqdtrB95zz6409pOKWqdKGk5lFjrxpINUY0bk%3D&amp;reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220104/b54db9b1/attachment.html>

From jayk123 at hotmail.com  Wed Jan  5 02:01:45 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 5 Jan 2022 01:01:45 +0000
Subject: [M3devel] GC scanning of register context
Message-ID: <MWHPR1401MB1951D0730FCCD2937D40F02CE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Mika's https://github.com/jaykrell/cm3/commit/cf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa

removed the scanning of register context in 2018.

< p(context, context + sizeof(ucontext_t));

> /*p(context, context + sizeof(ucontext_t));*/ /* cant be right */

Why?
I restored it yesterday.
I hadn't looked at blame and figured I had done it,
so was quicker go undo my own change, than perhaps
I should be of others.

The scan is a little dubious, imho:
 - With the right amount of volatile, it isn't needed.
   The registers will be on stack.
 - With a grow down stack, context should be on stack.

But:
 - With a grow up stack, it probably is not. It could be the end, depending on how the range is computed.
   HPPA is the only system I know with a grow up stack.
 - It seems kinda harmless, seems reasonable,
   and maybe some day we will have less volatile, maybe.
   Though really I want just cooperative suspend.

I will read the code more/again.

And: It looks like we acknowledge suspend requests before saving context.
sem_post before sigsuspend.
I will look at and fix that also.

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220105/d3d37d90/attachment-0001.html>

From lists at darko.org  Wed Jan  5 03:09:21 2022
From: lists at darko.org (Darko Volaric)
Date: Wed, 5 Jan 2022 03:09:21 +0100
Subject: [M3devel] Time to retire this mailing list?
Message-ID: <CAM3tuSH+fzOvy9vNtDSr7+K53uscrm3hyVz_4ROwo-WBvs9xFw@mail.gmail.com>

Almost all discussion has migrated to github which is easy to find and has
attracted new participants. Very few posts originate here, and those that
do will not be seen by the people on github.

This seems like a bad situation, at best it's confusing to new people
working on the project.

I propose the mailing list be shut down or that new posts are forbidden so
that all posts originate from github and are mirrored here.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220105/34287e7d/attachment.html>

From jayk123 at hotmail.com  Wed Jan  5 03:15:24 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 5 Jan 2022 02:15:24 +0000
Subject: [M3devel] GC scanning of register context
In-Reply-To: <MWHPR1401MB1951D0730FCCD2937D40F02CE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951D0730FCCD2937D40F02CE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951562F79A8DA89A4DE397EE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>

ok, the code is fairly convincing, but not entirely.

We get register context in the signal handler.
 That's how signal handling works (with SA_SIGINFO https://pubs.opengroup.org/onlinepubs/007904875/functions/sigaction.html).

It should be on the stack, whichever way it grows,
and we depend on that, as we use context as a stack pointer.
i.e. as the bottom of grow down stack.
(Unfortunately imho the Modula-3 termonology is backwards,
but that does not make the code wrong.)

I know NT exception handling works that way (context record
is on the stack). It kinda follows.
Since signals can nest, as long as you aren't using separate signal stack.
But even then, you have to wonder if the context is on the normal
stack or the signal stack. In any case, we depend on this.

In sigsuspend we save registers to stack.
This is the live registers of the signal handler.
These do not seem needed.
Well, there is the "me" root..but that appears untraced, and contains only untraced.

And we flush register windows.
I kinda imagine that is done earlier as part of signal handling anyway,
but we maybe check Linux/Solaris/BDS source. This is mainly for SPARC.
If signal handling does not previously flush register windows,
then we should do it, before sem_post.

And again, if the context is on the stack, which we depend on,
then scanning it is redundant for a grow down stack,
and for grow up the scan be merged, by adding sizeof(ucontext_t).

I'm really trying to fill in the IA64 support...I thought IA64
registers were stored on a grow up stack starting at the top of the
"normal" grow down stack, but Boehm GC looks more complicated..that the
register backing store can be anywhere.

 - Jay

________________________________
From: Jay K
Sent: Wednesday, January 5, 2022 1:01 AM
To: m3devel <m3devel at elegosoft.com>; Mika Nystrom <mika.nystrom at gmail.com>
Subject: GC scanning of register context

Mika's https://github.com/jaykrell/cm3/commit/cf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa

removed the scanning of register context in 2018.

< p(context, context + sizeof(ucontext_t));

> /*p(context, context + sizeof(ucontext_t));*/ /* cant be right */

Why?
I restored it yesterday.
I hadn't looked at blame and figured I had done it,
so was quicker go undo my own change, than perhaps
I should be of others.

The scan is a little dubious, imho:
 - With the right amount of volatile, it isn't needed.
   The registers will be on stack.
 - With a grow down stack, context should be on stack.

But:
 - With a grow up stack, it probably is not. It could be the end, depending on how the range is computed.
   HPPA is the only system I know with a grow up stack.
 - It seems kinda harmless, seems reasonable,
   and maybe some day we will have less volatile, maybe.
   Though really I want just cooperative suspend.

I will read the code more/again.

And: It looks like we acknowledge suspend requests before saving context.
sem_post before sigsuspend.
I will look at and fix that also.

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220105/3344a4e9/attachment.html>

From jayk123 at hotmail.com  Wed Jan  5 13:07:27 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 5 Jan 2022 12:07:27 +0000
Subject: [M3devel] GC scanning of register context
In-Reply-To: <CAE62oi=-wgGxj4-5zTS-MN2Va4dJ-bL16T3ey9XU3LBYbttSAw@mail.gmail.com>
References: <MWHPR1401MB1951D0730FCCD2937D40F02CE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oi=-wgGxj4-5zTS-MN2Va4dJ-bL16T3ey9XU3LBYbttSAw@mail.gmail.com>
Message-ID: <MWHPR1401MB19510DA001B6D753276B05FDE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>

The unconditional + makes sense (to me).
Context here are two valid non-contradictory meanings.
It is both a pointer to a ucontext_t, and a pointer to the stack, to approximately the most recently consumed part of the stack (to me "bottom", but the naming is subjective even if you assume a grow down x86 stack, as people will write addresses in either direction).
The context is on the stack.
The pointer is to the start of the context.
+ gets to the end of the context.

I have "fixed" it (within my limited understanding and testing..and I didn't test the coroutine paths).

If the stack is growing down, then the line is redundant.
If the stack is growing up, then it is maybe not redundant.
The only machine with a growing up stack is HPPA that nobody uses (I have a qemu command to try...and I ran it 10+ years ago..)

I fixed the other thing too.
I do understand what you were doing.
The code and the comments took me a few minutes.
You were wanting to use stuff on the stack after a function returned.
That is to be avoided at much cost.
I split the caller so that the returner now calls the rest.
Sorry, I went ahead and commited that, it had sat for a while.
If I broke it, sorry. We can still fix it.

Fwiw, I have IA64 (HP-UX) working now. ?
It had been in my head for a very long time as a system that would not "just work" due to the second stack for registers.
(Ok, I am missing a few lines for coroutines on IA64. I left it as Linux/amd64 anyway.)

 - Jay


________________________________
From: Mika Nystrom <mika.nystrom at gmail.com>
Sent: Wednesday, January 5, 2022 11:27 AM
To: Jay K <jayk123 at hotmail.com>
Cc: m3devel <m3devel at elegosoft.com>
Subject: Re: GC scanning of register context

The reason I removed it had to do with the stack direction, it didn't make sense to me that the stack can grow in either direction and the formula just has a "+".  I thought it was causing problems for me, it was scanning dead data in one of my tests of the coroutines.

I have seen you've asked me for reviews of code, but your requests are coinciding with back-to-work.  I will try to find some time to wrap my head around the design I committed a few years ago.  I did test it quite extensively but only on AMD64_LINUX, that's also why it's marked as such and not as portable code.

    Mika

On Tue, Jan 4, 2022 at 5:01 PM Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>> wrote:
Mika's https://github.com/jaykrell/cm3/commit/cf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjaykrell%2Fcm3%2Fcommit%2Fcf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa&data=04%7C01%7C%7Cbda2bb0854fa4a69cb0208d9d03e5141%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637769788331954984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1IX0cwzbD%2B7AjeqEzToTQPz3EKtekAKk6gx49qIqy3s%3D&reserved=0>

removed the scanning of register context in 2018.

< p(context, context + sizeof(ucontext_t));

> /*p(context, context + sizeof(ucontext_t));*/ /* cant be right */

Why?
I restored it yesterday.
I hadn't looked at blame and figured I had done it,
so was quicker go undo my own change, than perhaps
I should be of others.

The scan is a little dubious, imho:
 - With the right amount of volatile, it isn't needed.
   The registers will be on stack.
 - With a grow down stack, context should be on stack.

But:
 - With a grow up stack, it probably is not. It could be the end, depending on how the range is computed.
   HPPA is the only system I know with a grow up stack.
 - It seems kinda harmless, seems reasonable,
   and maybe some day we will have less volatile, maybe.
   Though really I want just cooperative suspend.

I will read the code more/again.

And: It looks like we acknowledge suspend requests before saving context.
sem_post before sigsuspend.
I will look at and fix that also.

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220105/d5b7eaf2/attachment-0001.html>

From rodney.bates at gmx.com  Wed Jan  5 23:35:37 2022
From: rodney.bates at gmx.com (Rodney Bates)
Date: Wed, 5 Jan 2022 16:35:37 -0600
Subject: [M3devel] GC scanning of register context
In-Reply-To: <MWHPR1401MB19510DA001B6D753276B05FDE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951D0730FCCD2937D40F02CE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oi=-wgGxj4-5zTS-MN2Va4dJ-bL16T3ey9XU3LBYbttSAw@mail.gmail.com>
 <MWHPR1401MB19510DA001B6D753276B05FDE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <311c7dbe-2f90-1f32-24ab-fd24af0f4a14@gmx.com>



On 1/5/22 6:07 AM, Jay K wrote:
> The unconditional + makes sense (to me).
> Context here are two valid non-contradictory meanings.
> It is both a pointer to a ucontext_t, and a pointer to the stack, to approximately the most recently consumed part of the stack (to me "bottom", but the naming is subjective even if you assume a grow down x86 stack, as people will write addresses in either direction).

I'm not taking a position, but I'm curious about your sense
of what constitutes up and down, in light of the, ahem,
"intriguing" variety of usages.? Abstractly, most consider stack
top to be most-recently-pushed.? Except, for gdb, (and inherited
by m3gdb), where to move toward most-recently-pushed, you need
the 'down' command.

But at machine level, up often means toward higher-numbered
addresses, which might be towards most-recently-pushed or
the other way.

But in a picture, higher addresses are usually toward the
bottom of the page.? But I have seen all 4 combinations of
which direction addresses increase on the page and which
direction a stack grows on the page.? I have even seen at
least one case where addresses increase towards the top of
the page *between* pushed items, but towards the bottom
*within* a multi-word item on the stack such as a double
word scalar, or an array or record.? Or something like that, where
address increases jump in alternating directions on the page.

> The context is on the stack.
> The pointer is to the start of the context.
> + gets to the end of the context.
>
> I have "fixed" it (within my limited understanding and testing..and I didn't test the coroutine paths).
>
> If the stack is growing down, then the line is redundant.
> If the stack is growing up, then it is maybe not redundant.
> The only machine with a growing up stack is HPPA that nobody uses (I have a qemu command to try...and I ran it 10+ years ago..)
>
> I fixed the other thing too.
> I do understand what you were doing.
> The code and the comments took me a few minutes.
> You were wanting to use stuff on the stack after a function returned.
> That is to be avoided at much cost.
> I split the caller so that the returner now calls the rest.
> Sorry, I went ahead and commited that, it had sat for a while.
> If I broke it, sorry. We can still fix it.
>
> Fwiw, I have IA64 (HP-UX) working now. ?
> It had been in my head for a very long time as a system that would not "just work" due to the second stack for registers.
> (Ok, I am missing a few lines for coroutines on IA64. I left it as Linux/amd64 anyway.)
>
> ?- Jay
>
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> *From:* Mika Nystrom <mika.nystrom at gmail.com>
> *Sent:* Wednesday, January 5, 2022 11:27 AM
> *To:* Jay K <jayk123 at hotmail.com>
> *Cc:* m3devel <m3devel at elegosoft.com>
> *Subject:* Re: GC scanning of register context
> The reason I removed it had to do with the stack direction, it didn't make sense to me that the stack can grow in either direction and the formula just has a "+".? I thought it was causing problems for me, it was scanning dead data in one of my tests of the coroutines.
>
> I have seen you've asked me for reviews of code, but your requests are coinciding with back-to-work.? I will try to find some time to wrap my head around the design I committed a few years ago.? I did test it quite extensively but only on AMD64_LINUX, that's also why it's marked as such and not as portable code.
>
> ? ? Mika
>
> On Tue, Jan 4, 2022 at 5:01 PM Jay K <jayk123 at hotmail.com <mailto:jayk123 at hotmail.com>> wrote:
>
>     Mika's https://github.com/jaykrell/cm3/commit/cf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjaykrell%2Fcm3%2Fcommit%2Fcf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa&data=04%7C01%7C%7Cbda2bb0854fa4a69cb0208d9d03e5141%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637769788331954984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1IX0cwzbD%2B7AjeqEzToTQPz3EKtekAKk6gx49qIqy3s%3D&reserved=0>
>
>     removed the scanning of register context in 2018.
>
>     < p(context, context + sizeof(ucontext_t));
>
>     > /*p(context, context + sizeof(ucontext_t));*/ /* cant be right */
>
>     Why?
>     I restored it yesterday.
>     I hadn't looked at blame and figured I had done it,
>     so was quicker go undo my own change, than perhaps
>     I should be of others.
>
>     The scan is a little dubious, imho:
>     ?- With the right amount of volatile, it isn't needed.
>     ? ?The registers will be on stack.
>     ?- With a grow down stack, context should be on stack.
>
>     But:
>     ?- With a grow up stack, it probably is not. It could be the end, depending on how the range is computed.
>     ? ?HPPA is the only system I know with a grow up stack.
>     ?- It seems kinda harmless, seems reasonable,
>     ? ?and maybe some day we will have less volatile, maybe.
>     ? ?Though really I want just cooperative suspend.
>
>     I will read the code more/again.
>
>     And: It looks like we acknowledge suspend requests before saving context.
>     sem_post before sigsuspend.
>     I will look at and fix that also.
>
>     ?- Jay
>
>
> _______________________________________________
> M3devel mailing list
> M3devel at elegosoft.com
> https://m3lists.elegosoft.com/mailman/listinfo/m3devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220105/889943af/attachment.html>

From jayk123 at hotmail.com  Thu Jan  6 00:39:55 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 5 Jan 2022 23:39:55 +0000
Subject: [M3devel] GC scanning of register context
In-Reply-To: <311c7dbe-2f90-1f32-24ab-fd24af0f4a14@gmx.com>
References: <MWHPR1401MB1951D0730FCCD2937D40F02CE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oi=-wgGxj4-5zTS-MN2Va4dJ-bL16T3ey9XU3LBYbttSAw@mail.gmail.com>
 <MWHPR1401MB19510DA001B6D753276B05FDE64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <311c7dbe-2f90-1f32-24ab-fd24af0f4a14@gmx.com>
Message-ID: <MWHPR1401MB1951B77EF007EC1C32B8B613E64B9@MWHPR1401MB1951.namprd14.prod.outlook.com>

 > But I have seen all 4 combinations of

Exactly: Everything makes sense.

I know it is whacky but I *mildly* favor several views depending on slightly different context.
Mildly, but enough that I renamed the variables. Hopefully nobody minds and
it does not cause/increase confusion.

I understand the semantics are not changed by the pictures.
I understand having varying pictures and names can be confusing.
If anyone feels strongly we can rename things.

When I view memory in a debugger, addresses go:
 0 low memory, bottom
 1
 2
 3 high memory, top

That isn't up to me. That is the debugger.
And it generally makes sense. At least for data.

and since we really treating the stack like data here, these
are the terms I favor here.

When I view stacks in a debugger, they go:
 Child-Sp RetAddr
        0 WriteFile    top of conceptual stack, printed on top, at low address
        1 printf
        2 main
        3 mainCRTStartup

That isn't up to me. That is the debugger.

and I copy/paste that sometimes into email/comments

The debugger can navigate frames with .f+ and .f- to go up and down frames.
I get confused sometimes.

When I take plates or trays off a pile in the kitchen/cafeteria,
or play Towers Of Hanoi:

   -    top
  ---
 -----

When I think of pushing return addresses or parameters/local though
for some reason I think of backwards from the debugger:
 3 mainCRTStartup  top, highest address
 2 main
 1 printf
 0 WriteFile       bottom, lowest address

It can also work left to right, because, English.
 0 1 2 3

Windows stacks have three+ associated address:
   0x12340000   "base", page boundary, top, inaccessible
   0x12...000   "limit", lowest touched address, commited
   0x11...fff   limit-1, first untouched address, guard page; this isn't stored, just -1 from prior
   0x1.......   guard page(s), I think these are below limit; this isn't stored publically but there is VirtualQuery and SetThreadStackGuarantee,
                I believe the point of which is, to allow for more than one page to handle
                the stack overflow exception when you are getting low
   0x10000000   AllocationBase, what was VirtualAlloc'ed and to be VirtualFreed


I bring this up because there are those names "base" and "limit", but
I am not using them.
Some Solaris APIs swapped the meaning of "base" at some point (see m3core.h).

  0:000> !teb
    StackBase:            0000008c30f30000
    StackLimit:           0000008c30e34000

  0:000> !vprot 0000008c30f30000
  BaseAddress:       0000008c30f30000
  AllocationBase:    0000000000000000
  State:             00010000  MEM_FREE
  Protect:           00000001  PAGE_NOACCESS

  0:000> !vprot 0000008c30e34000
  BaseAddress:       0000008c30e34000
  AllocationBase:    0000008c30e30000
  State:             00001000  MEM_COMMIT
  Protect:           00000004  PAGE_READWRITE

  0:000> !vprot 0000008c30e34000-1
  State:             00001000  MEM_COMMIT
  Protect:           00000104  PAGE_READWRITE + PAGE_GUARD

  0:000> !vprot 0000008c30e30000
  BaseAddress:       0000008c30e30000
  AllocationBase:    0000008c30e30000
  State:             00002000  MEM_RESERVE

 Ok I was wrong. The guard pages are commited.
 That makes sense. "Commit" means can be touched
 even under low memory, as long as paging device is working (and physical memory).
 But Limit is the where the guard page boundary is.

 Given that PARISC is the only system I know with a grow up stack,
 I don't think we will see a grow up stack in our children's lifetimes.

 But at least with WebAssembly the return address stack is not accessible,
 and the evaluation stack is only conceptual.

 It does make you wonder though...if x86 could use a growup stack through "software".
 I mean, you don't actually have to use call and ret, or push and pop.
 You can use mov, jmp, add, sub. call and ret are "just" huffman compression/macros..

Possibly more so with the architectures, that use link register for return
and lack push/pop. I guess it is more about ABI than instruction set?

 - Jay

________________________________
From: Rodney Bates <rodney.bates at gmx.com>
Sent: Wednesday, January 5, 2022 10:35 PM
To: Jay K <jayk123 at hotmail.com>; Mika Nystrom <mika.nystrom at gmail.com>
Cc: m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] GC scanning of register context



On 1/5/22 6:07 AM, Jay K wrote:
The unconditional + makes sense (to me).
Context here are two valid non-contradictory meanings.
It is both a pointer to a ucontext_t, and a pointer to the stack, to approximately the most recently consumed part of the stack (to me "bottom", but the naming is subjective even if you assume a grow down x86 stack, as people will write addresses in either direction).

I'm not taking a position, but I'm curious about your sense
of what constitutes up and down, in light of the, ahem,
"intriguing" variety of usages.  Abstractly, most consider stack
top to be most-recently-pushed.  Except, for gdb, (and inherited
by m3gdb), where to move toward most-recently-pushed, you need
the 'down' command.

But at machine level, up often means toward higher-numbered
addresses, which might be towards most-recently-pushed or
the other way.

But in a picture, higher addresses are usually toward the
bottom of the page.  But I have seen all 4 combinations of
which direction addresses increase on the page and which
direction a stack grows on the page.  I have even seen at
least one case where addresses increase towards the top of
the page *between* pushed items, but towards the bottom
*within* a multi-word item on the stack such as a double
word scalar, or an array or record.  Or something like that, where
address increases jump in alternating directions on the page.

The context is on the stack.
The pointer is to the start of the context.
+ gets to the end of the context.

I have "fixed" it (within my limited understanding and testing..and I didn't test the coroutine paths).

If the stack is growing down, then the line is redundant.
If the stack is growing up, then it is maybe not redundant.
The only machine with a growing up stack is HPPA that nobody uses (I have a qemu command to try...and I ran it 10+ years ago..)

I fixed the other thing too.
I do understand what you were doing.
The code and the comments took me a few minutes.
You were wanting to use stuff on the stack after a function returned.
That is to be avoided at much cost.
I split the caller so that the returner now calls the rest.
Sorry, I went ahead and commited that, it had sat for a while.
If I broke it, sorry. We can still fix it.

Fwiw, I have IA64 (HP-UX) working now. ?
It had been in my head for a very long time as a system that would not "just work" due to the second stack for registers.
(Ok, I am missing a few lines for coroutines on IA64. I left it as Linux/amd64 anyway.)

 - Jay


________________________________
From: Mika Nystrom <mika.nystrom at gmail.com><mailto:mika.nystrom at gmail.com>
Sent: Wednesday, January 5, 2022 11:27 AM
To: Jay K <jayk123 at hotmail.com><mailto:jayk123 at hotmail.com>
Cc: m3devel <m3devel at elegosoft.com><mailto:m3devel at elegosoft.com>
Subject: Re: GC scanning of register context

The reason I removed it had to do with the stack direction, it didn't make sense to me that the stack can grow in either direction and the formula just has a "+".  I thought it was causing problems for me, it was scanning dead data in one of my tests of the coroutines.

I have seen you've asked me for reviews of code, but your requests are coinciding with back-to-work.  I will try to find some time to wrap my head around the design I committed a few years ago.  I did test it quite extensively but only on AMD64_LINUX, that's also why it's marked as such and not as portable code.

    Mika

On Tue, Jan 4, 2022 at 5:01 PM Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>> wrote:
Mika's https://github.com/jaykrell/cm3/commit/cf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjaykrell%2Fcm3%2Fcommit%2Fcf9ec9fcb167bd53c0eda4bcf52d5d9da40053aa&data=04%7C01%7C%7C87159ba96a2d42d3a0e908d9d09bb709%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770189475781619%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wOfl4DfBn2fHFsd3TD1AGmRZC8YVesYTlwG%2BOCpm0is%3D&reserved=0>

removed the scanning of register context in 2018.

< p(context, context + sizeof(ucontext_t));

> /*p(context, context + sizeof(ucontext_t));*/ /* cant be right */

Why?
I restored it yesterday.
I hadn't looked at blame and figured I had done it,
so was quicker go undo my own change, than perhaps
I should be of others.

The scan is a little dubious, imho:
 - With the right amount of volatile, it isn't needed.
   The registers will be on stack.
 - With a grow down stack, context should be on stack.

But:
 - With a grow up stack, it probably is not. It could be the end, depending on how the range is computed.
   HPPA is the only system I know with a grow up stack.
 - It seems kinda harmless, seems reasonable,
   and maybe some day we will have less volatile, maybe.
   Though really I want just cooperative suspend.

I will read the code more/again.

And: It looks like we acknowledge suspend requests before saving context.
sem_post before sigsuspend.
I will look at and fix that also.

 - Jay



_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C87159ba96a2d42d3a0e908d9d09bb709%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770189475781619%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=lMbGbOyKTM3VdcXMD1sZcHdsNA%2FHHtrULwBAnKaQBec%3D&reserved=0>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220105/283d393d/attachment-0001.html>

From jayk123 at hotmail.com  Thu Jan  6 08:24:23 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 07:24:23 +0000
Subject: [M3devel] reference to variable exceeds its storage space
Message-ID: <MWHPR1401MB1951EFC5C5DB238BBA7CC12EE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

fyi:
C:\s\cm3\scripts\python\src\hash\MD5.m3(277) : warning C4739: reference to variable 'MD5_m_212_L_213' exceeds its storage space

    len64 := LOOPHOLE(VAL(ctrl.length + ctrl.bufLen,LONGINT) * 8L,Int64On32);

  Int64On32 = RECORD a, b: Int32;  END;

((*(INT64*)(&MD5_m_212_L_213))=(INT64)( ((INT64)(((INT64)( /*LH2*/ (INT64)((INT32)( ((INT32)(*((UINT32*)(4+((ADDRESS)(ctrl_L_114))))))+ ((INT32)(*((UINT32*)(ctrl_L_114))))))))*  INT64_(8)))));

 /* Var_Type2 */ ADDRESS MD5_m_212_L_213 = { 0 };

 /* Var_Type3 */ STRUCT(8) MD5_m_214_L_215 = { 0 };

..not sure yet if this is an m3c bug or m3front bug or what..not enough IR tracing in M3C..

Only a problem on 32bit targets.

I have local changes to m3c but I think this is an m3front bug. Investigating..

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/e69032be/attachment.html>

From jayk123 at hotmail.com  Thu Jan  6 08:47:09 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 07:47:09 +0000
Subject: [M3devel] reference to variable exceeds its storage space
In-Reply-To: <MWHPR1401MB1951EFC5C5DB238BBA7CC12EE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951EFC5C5DB238BBA7CC12EE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB195179FE2C18CF0DF8FF5030E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

I added an assert, I think a correct one:

    *** runtime error:
    ***    <*ASSERT*> failed: type = CGType.Struct OR byte_size = CG_Bytes [type]
    ***    file "../src/M3CG_MultiPass.m3", line 383

 cm3!RTHooks__AssertFailed+0xbf         [m3core\src\runtime\common\RTHooks.m3 @ 129]
 cm3!M3CG_MultiPass__declare_temp+0x22b [m3middle\src\M3CG_MultiPass.m3 @ 384]
 cm3!M3CG_Check__declare_temp+0x116     [m3middle\src\M3CG_Check.m3 @ 417]
 cm3!CG__Declare_temp+0x207             [m3front\src\misc\CG.m3 @ 533]
 cm3!CastExpr__Prep+0x916               [m3front\src\exprs\CastExpr.m3 @ 247]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!Loophole__Prep+0x157               [m3front\src\builtinOps\Loophole.m3 @ 56]
 cm3!CallExpr__Prep+0xe9                [m3front\src\exprs\CallExpr.m3 @ 375]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!AssignStmt__PrepForEmit+0xbc       [m3front\src\stmts\AssignStmt.m3 @ 286]
 cm3!AssignStmt__Compile+0x281          [m3front\src\stmts\AssignStmt.m3 @ 838]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!BlockStmt__Compile+0x455           [m3front\src\stmts\BlockStmt.m3 @ 103]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!Procedure__GenBody+0x620           [m3front\src\values\Procedure.m3 @ 671]
 cm3!Procedure__EmitBody+0xb8           [m3front\src\values\Procedure.m3 @ 636]
 cm3!ProcBody__EmitBody+0x340           [m3front\src\misc\ProcBody.m3 @ 171]
 cm3!ProcBody__EmitAll+0x140            [m3front\src\misc\ProcBody.m3 @ 65]
 cm3!Module__CompileModule+0x2c1        [m3front\src\values\Module.m3 @ 1009]
 cm3!Module__Compile+0x3b5              [m3front\src\values\Module.m3 @ 940]
 cm3!M3Front__DoCompile+0xb61           [m3front\src\misc\M3Front.m3 @ 220]

(Unfortunately asserts in m3c are not useful, since m3cg_multipass runs though successfully recording all the state,
so then when m3c runs, you cannot see where m3front produced the bad data. This is the nature of iterating over long lived data structures instead of single pass w/o memory. But m3cg_multipass is good.)

I think this is an m3front bug, confusing byvalue and byref with structs or something.

 - Jay

________________________________
From: Jay K
Sent: Thursday, January 6, 2022 7:24 AM
To: m3devel <m3devel at elegosoft.com>
Subject: reference to variable exceeds its storage space

fyi:
C:\s\cm3\scripts\python\src\hash\MD5.m3(277) : warning C4739: reference to variable 'MD5_m_212_L_213' exceeds its storage space

    len64 := LOOPHOLE(VAL(ctrl.length + ctrl.bufLen,LONGINT) * 8L,Int64On32);

  Int64On32 = RECORD a, b: Int32;  END;

((*(INT64*)(&MD5_m_212_L_213))=(INT64)( ((INT64)(((INT64)( /*LH2*/ (INT64)((INT32)( ((INT32)(*((UINT32*)(4+((ADDRESS)(ctrl_L_114))))))+ ((INT32)(*((UINT32*)(ctrl_L_114))))))))*  INT64_(8)))));

 /* Var_Type2 */ ADDRESS MD5_m_212_L_213 = { 0 };

 /* Var_Type3 */ STRUCT(8) MD5_m_214_L_215 = { 0 };

..not sure yet if this is an m3c bug or m3front bug or what..not enough IR tracing in M3C..

Only a problem on 32bit targets.

I have local changes to m3c but I think this is an m3front bug. Investigating..

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/23556575/attachment.html>

From jayk123 at hotmail.com  Thu Jan  6 08:58:32 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 07:58:32 +0000
Subject: [M3devel] reference to variable exceeds its storage space
In-Reply-To: <MWHPR1401MB195179FE2C18CF0DF8FF5030E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951EFC5C5DB238BBA7CC12EE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB195179FE2C18CF0DF8FF5030E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951E3A22E609B376D75ED26E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Just reading the code, not stepping:

C:\s\cm3\m3-sys\m3front\src\types\RecordType.m3(172):    p.info.stk_type  := CG.Type.Addr;

isn't that kinda wierd?
I mean, I get the records are variably sized and difficult to represent, but I'd hope for an abstraction here and leave it to the backend.

I could imagine something like, allocate a temporary of the size requested, and then another temporary that points to it.
But then the level of indirection I think would be incorrect.

Another thing to try is to ignore the type and just pay attention to the size.
"all temporaries are aligned arrays of a number of bytes, or heck, all variables are such, no need for int, float, etc."
I mean, you know, passing along type/size tuples is generally redundant, except for records, or maybe arrays, if arrays work "that way".

I think stk_type should be .Struct.

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:47 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: reference to variable exceeds its storage space

I added an assert, I think a correct one:

    *** runtime error:
    ***    <*ASSERT*> failed: type = CGType.Struct OR byte_size = CG_Bytes [type]
    ***    file "../src/M3CG_MultiPass.m3", line 383

 cm3!RTHooks__AssertFailed+0xbf         [m3core\src\runtime\common\RTHooks.m3 @ 129]
 cm3!M3CG_MultiPass__declare_temp+0x22b [m3middle\src\M3CG_MultiPass.m3 @ 384]
 cm3!M3CG_Check__declare_temp+0x116     [m3middle\src\M3CG_Check.m3 @ 417]
 cm3!CG__Declare_temp+0x207             [m3front\src\misc\CG.m3 @ 533]
 cm3!CastExpr__Prep+0x916               [m3front\src\exprs\CastExpr.m3 @ 247]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!Loophole__Prep+0x157               [m3front\src\builtinOps\Loophole.m3 @ 56]
 cm3!CallExpr__Prep+0xe9                [m3front\src\exprs\CallExpr.m3 @ 375]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!AssignStmt__PrepForEmit+0xbc       [m3front\src\stmts\AssignStmt.m3 @ 286]
 cm3!AssignStmt__Compile+0x281          [m3front\src\stmts\AssignStmt.m3 @ 838]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!BlockStmt__Compile+0x455           [m3front\src\stmts\BlockStmt.m3 @ 103]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!Procedure__GenBody+0x620           [m3front\src\values\Procedure.m3 @ 671]
 cm3!Procedure__EmitBody+0xb8           [m3front\src\values\Procedure.m3 @ 636]
 cm3!ProcBody__EmitBody+0x340           [m3front\src\misc\ProcBody.m3 @ 171]
 cm3!ProcBody__EmitAll+0x140            [m3front\src\misc\ProcBody.m3 @ 65]
 cm3!Module__CompileModule+0x2c1        [m3front\src\values\Module.m3 @ 1009]
 cm3!Module__Compile+0x3b5              [m3front\src\values\Module.m3 @ 940]
 cm3!M3Front__DoCompile+0xb61           [m3front\src\misc\M3Front.m3 @ 220]

(Unfortunately asserts in m3c are not useful, since m3cg_multipass runs though successfully recording all the state,
so then when m3c runs, you cannot see where m3front produced the bad data. This is the nature of iterating over long lived data structures instead of single pass w/o memory. But m3cg_multipass is good.)

I think this is an m3front bug, confusing byvalue and byref with structs or something.

 - Jay

________________________________
From: Jay K
Sent: Thursday, January 6, 2022 7:24 AM
To: m3devel <m3devel at elegosoft.com>
Subject: reference to variable exceeds its storage space

fyi:
C:\s\cm3\scripts\python\src\hash\MD5.m3(277) : warning C4739: reference to variable 'MD5_m_212_L_213' exceeds its storage space

    len64 := LOOPHOLE(VAL(ctrl.length + ctrl.bufLen,LONGINT) * 8L,Int64On32);

  Int64On32 = RECORD a, b: Int32;  END;

((*(INT64*)(&MD5_m_212_L_213))=(INT64)( ((INT64)(((INT64)( /*LH2*/ (INT64)((INT32)( ((INT32)(*((UINT32*)(4+((ADDRESS)(ctrl_L_114))))))+ ((INT32)(*((UINT32*)(ctrl_L_114))))))))*  INT64_(8)))));

 /* Var_Type2 */ ADDRESS MD5_m_212_L_213 = { 0 };

 /* Var_Type3 */ STRUCT(8) MD5_m_214_L_215 = { 0 };

..not sure yet if this is an m3c bug or m3front bug or what..not enough IR tracing in M3C..

Only a problem on 32bit targets.

I have local changes to m3c but I think this is an m3front bug. Investigating..

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/55aac62a/attachment-0001.html>

From jayk123 at hotmail.com  Thu Jan  6 11:15:16 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 10:15:16 +0000
Subject: [M3devel] reference to variable exceeds its storage space
In-Reply-To: <MWHPR1401MB1951E3A22E609B376D75ED26E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951EFC5C5DB238BBA7CC12EE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB195179FE2C18CF0DF8FF5030E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951E3A22E609B376D75ED26E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB19518D727E7E59939DF8A3DEE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Rodney?

m3front: Change RecordTypes stack type from Addr to Struct. by jaykrell ? Pull Request #829 ? modula3/cm3 (github.com)<https://github.com/modula3/cm3/pull/829>

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:58 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: reference to variable exceeds its storage space

Just reading the code, not stepping:

C:\s\cm3\m3-sys\m3front\src\types\RecordType.m3(172):    p.info.stk_type  := CG.Type.Addr;

isn't that kinda wierd?
I mean, I get the records are variably sized and difficult to represent, but I'd hope for an abstraction here and leave it to the backend.

I could imagine something like, allocate a temporary of the size requested, and then another temporary that points to it.
But then the level of indirection I think would be incorrect.

Another thing to try is to ignore the type and just pay attention to the size.
"all temporaries are aligned arrays of a number of bytes, or heck, all variables are such, no need for int, float, etc."
I mean, you know, passing along type/size tuples is generally redundant, except for records, or maybe arrays, if arrays work "that way".

I think stk_type should be .Struct.

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:47 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: reference to variable exceeds its storage space

I added an assert, I think a correct one:

    *** runtime error:
    ***    <*ASSERT*> failed: type = CGType.Struct OR byte_size = CG_Bytes [type]
    ***    file "../src/M3CG_MultiPass.m3", line 383

 cm3!RTHooks__AssertFailed+0xbf         [m3core\src\runtime\common\RTHooks.m3 @ 129]
 cm3!M3CG_MultiPass__declare_temp+0x22b [m3middle\src\M3CG_MultiPass.m3 @ 384]
 cm3!M3CG_Check__declare_temp+0x116     [m3middle\src\M3CG_Check.m3 @ 417]
 cm3!CG__Declare_temp+0x207             [m3front\src\misc\CG.m3 @ 533]
 cm3!CastExpr__Prep+0x916               [m3front\src\exprs\CastExpr.m3 @ 247]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!Loophole__Prep+0x157               [m3front\src\builtinOps\Loophole.m3 @ 56]
 cm3!CallExpr__Prep+0xe9                [m3front\src\exprs\CallExpr.m3 @ 375]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!AssignStmt__PrepForEmit+0xbc       [m3front\src\stmts\AssignStmt.m3 @ 286]
 cm3!AssignStmt__Compile+0x281          [m3front\src\stmts\AssignStmt.m3 @ 838]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!BlockStmt__Compile+0x455           [m3front\src\stmts\BlockStmt.m3 @ 103]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!Procedure__GenBody+0x620           [m3front\src\values\Procedure.m3 @ 671]
 cm3!Procedure__EmitBody+0xb8           [m3front\src\values\Procedure.m3 @ 636]
 cm3!ProcBody__EmitBody+0x340           [m3front\src\misc\ProcBody.m3 @ 171]
 cm3!ProcBody__EmitAll+0x140            [m3front\src\misc\ProcBody.m3 @ 65]
 cm3!Module__CompileModule+0x2c1        [m3front\src\values\Module.m3 @ 1009]
 cm3!Module__Compile+0x3b5              [m3front\src\values\Module.m3 @ 940]
 cm3!M3Front__DoCompile+0xb61           [m3front\src\misc\M3Front.m3 @ 220]

(Unfortunately asserts in m3c are not useful, since m3cg_multipass runs though successfully recording all the state,
so then when m3c runs, you cannot see where m3front produced the bad data. This is the nature of iterating over long lived data structures instead of single pass w/o memory. But m3cg_multipass is good.)

I think this is an m3front bug, confusing byvalue and byref with structs or something.

 - Jay

________________________________
From: Jay K
Sent: Thursday, January 6, 2022 7:24 AM
To: m3devel <m3devel at elegosoft.com>
Subject: reference to variable exceeds its storage space

fyi:
C:\s\cm3\scripts\python\src\hash\MD5.m3(277) : warning C4739: reference to variable 'MD5_m_212_L_213' exceeds its storage space

    len64 := LOOPHOLE(VAL(ctrl.length + ctrl.bufLen,LONGINT) * 8L,Int64On32);

  Int64On32 = RECORD a, b: Int32;  END;

((*(INT64*)(&MD5_m_212_L_213))=(INT64)( ((INT64)(((INT64)( /*LH2*/ (INT64)((INT32)( ((INT32)(*((UINT32*)(4+((ADDRESS)(ctrl_L_114))))))+ ((INT32)(*((UINT32*)(ctrl_L_114))))))))*  INT64_(8)))));

 /* Var_Type2 */ ADDRESS MD5_m_212_L_213 = { 0 };

 /* Var_Type3 */ STRUCT(8) MD5_m_214_L_215 = { 0 };

..not sure yet if this is an m3c bug or m3front bug or what..not enough IR tracing in M3C..

Only a problem on 32bit targets.

I have local changes to m3c but I think this is an m3front bug. Investigating..

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/a4b63103/attachment.html>

From jayk123 at hotmail.com  Thu Jan  6 17:39:31 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 16:39:31 +0000
Subject: [M3devel] making CI "optionally required"?
Message-ID: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

On a PR by PR basis, can I make checks required? Possibly check by check?
Like, "complete once checks pass", cause I am impatient and want to click it now, and have it wait/poll, not me.

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/dd1be547/attachment-0001.html>

From coder5506 at pobox.com  Thu Jan  6 19:09:19 2022
From: coder5506 at pobox.com (Eric Sessoms)
Date: Thu, 06 Jan 2022 13:09:19 -0500
Subject: [M3devel] making CI "optionally required"?
In-Reply-To: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>

There's two steps.  First you have to enable "auto-merge" for the repository.  I just did that, it's on the github settings page for CM3.  Then you can set an individual PR to auto-merge.  (Supposedly. I haven't used it.)

See https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/automatically-merging-a-pull-request

Given the way you've been working, you might want to play with the concurrency options.  See https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#concurrency

Edit the file .github/workflows/integration.yml to change it.

In particular, you might want to use the option to cancel an in-progress run.  I recommend you don't remove the concurrency controls altogether, because that's a quick way to get Github to block your access to CI.

For anything more fine-grained, I think you'd have to start editing the workflow files.  For example, when I've wanted to run just one check I've just commented out the ones I didn't want.  You can get fancier, but then you're writing more code.

On Thu, Jan 6, 2022, at 11:39 AM, Jay K wrote:
> On a PR by PR basis, can I make checks required? Possibly check by 
> check?
> Like, "complete once checks pass", cause I am impatient and want to 
> click it now, and have it wait/poll, not me.
>
>  - Jay

From jayk123 at hotmail.com  Thu Jan  6 20:13:32 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 19:13:32 +0000
Subject: [M3devel] making CI "optionally required"?
In-Reply-To: <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>
References: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>
Message-ID: <MWHPR1401MB1951CF27553B69BD4F5A2CEFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Um, now nothing runs?

 - Jay

________________________________
From: Eric Sessoms <coder5506 at pobox.com>
Sent: Thursday, January 6, 2022 6:09 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: making CI "optionally required"?

There's two steps.  First you have to enable "auto-merge" for the repository.  I just did that, it's on the github settings page for CM3.  Then you can set an individual PR to auto-merge.  (Supposedly. I haven't used it.)

See https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Fpull-requests%2Fcollaborating-with-pull-requests%2Fincorporating-changes-from-a-pull-request%2Fautomatically-merging-a-pull-request&amp;data=04%7C01%7C%7Cacb40d9a4bc64ec7cc3208d9d13fd009%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770894290626734%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=YT11VfKCziMOqQoZ5xl0ksAaggzuRMlRP%2B1H3cjUTrQ%3D&amp;reserved=0

Given the way you've been working, you might want to play with the concurrency options.  See https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fworkflow-syntax-for-github-actions%23concurrency&amp;data=04%7C01%7C%7Cacb40d9a4bc64ec7cc3208d9d13fd009%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770894290626734%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=JRt67aJINU7zSMKFgGjmDedxKyAQRLMH6V015ONCSOA%3D&amp;reserved=0

Edit the file .github/workflows/integration.yml to change it.

In particular, you might want to use the option to cancel an in-progress run.  I recommend you don't remove the concurrency controls altogether, because that's a quick way to get Github to block your access to CI.

For anything more fine-grained, I think you'd have to start editing the workflow files.  For example, when I've wanted to run just one check I've just commented out the ones I didn't want.  You can get fancier, but then you're writing more code.

On Thu, Jan 6, 2022, at 11:39 AM, Jay K wrote:
> On a PR by PR basis, can I make checks required? Possibly check by
> check?
> Like, "complete once checks pass", cause I am impatient and want to
> click it now, and have it wait/poll, not me.
>
>  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/d17c6dfa/attachment.html>

From jayk123 at hotmail.com  Thu Jan  6 20:16:10 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 19:16:10 +0000
Subject: [M3devel] reference to variable exceeds its storage space
In-Reply-To: <MWHPR1401MB19518D727E7E59939DF8A3DEE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951EFC5C5DB238BBA7CC12EE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB195179FE2C18CF0DF8FF5030E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951E3A22E609B376D75ED26E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB19518D727E7E59939DF8A3DEE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951AA84AB7F29073DBC9431E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

er sorry:
m3front: Change RecordTypes stack type from Addr to Struct. by jaykrell ? Pull Request #836 ? modula3/cm3 (github.com)<https://github.com/modula3/cm3/pull/836>

I checked in a workaround:
m3c: Workaround m3front asking for types smaller than their size. by jaykrell ? Pull Request #837 ? modula3/cm3 (github.com)<https://github.com/modula3/cm3/pull/837>


but I think m3front needs to be fixed here.

 - Jay


________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 10:15 AM
To: m3devel <m3devel at elegosoft.com>; rodney.m.bates at acm.org <rodney.m.bates at acm.org>
Subject: Re: reference to variable exceeds its storage space

Rodney?

m3front: Change RecordTypes stack type from Addr to Struct. by jaykrell ? Pull Request #829 ? modula3/cm3 (github.com)<https://github.com/modula3/cm3/pull/829>

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:58 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: reference to variable exceeds its storage space

Just reading the code, not stepping:

C:\s\cm3\m3-sys\m3front\src\types\RecordType.m3(172):    p.info.stk_type  := CG.Type.Addr;

isn't that kinda wierd?
I mean, I get the records are variably sized and difficult to represent, but I'd hope for an abstraction here and leave it to the backend.

I could imagine something like, allocate a temporary of the size requested, and then another temporary that points to it.
But then the level of indirection I think would be incorrect.

Another thing to try is to ignore the type and just pay attention to the size.
"all temporaries are aligned arrays of a number of bytes, or heck, all variables are such, no need for int, float, etc."
I mean, you know, passing along type/size tuples is generally redundant, except for records, or maybe arrays, if arrays work "that way".

I think stk_type should be .Struct.

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:47 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: reference to variable exceeds its storage space

I added an assert, I think a correct one:

    *** runtime error:
    ***    <*ASSERT*> failed: type = CGType.Struct OR byte_size = CG_Bytes [type]
    ***    file "../src/M3CG_MultiPass.m3", line 383

 cm3!RTHooks__AssertFailed+0xbf         [m3core\src\runtime\common\RTHooks.m3 @ 129]
 cm3!M3CG_MultiPass__declare_temp+0x22b [m3middle\src\M3CG_MultiPass.m3 @ 384]
 cm3!M3CG_Check__declare_temp+0x116     [m3middle\src\M3CG_Check.m3 @ 417]
 cm3!CG__Declare_temp+0x207             [m3front\src\misc\CG.m3 @ 533]
 cm3!CastExpr__Prep+0x916               [m3front\src\exprs\CastExpr.m3 @ 247]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!Loophole__Prep+0x157               [m3front\src\builtinOps\Loophole.m3 @ 56]
 cm3!CallExpr__Prep+0xe9                [m3front\src\exprs\CallExpr.m3 @ 375]
 cm3!Expr__Prep+0x9a                    [m3front\src\exprs\Expr.m3 @ 362]
 cm3!AssignStmt__PrepForEmit+0xbc       [m3front\src\stmts\AssignStmt.m3 @ 286]
 cm3!AssignStmt__Compile+0x281          [m3front\src\stmts\AssignStmt.m3 @ 838]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!BlockStmt__Compile+0x455           [m3front\src\stmts\BlockStmt.m3 @ 103]
 cm3!Stmt__Compile+0xb1                 [m3front\src\stmts\Stmt.m3 @ 115]
 cm3!Procedure__GenBody+0x620           [m3front\src\values\Procedure.m3 @ 671]
 cm3!Procedure__EmitBody+0xb8           [m3front\src\values\Procedure.m3 @ 636]
 cm3!ProcBody__EmitBody+0x340           [m3front\src\misc\ProcBody.m3 @ 171]
 cm3!ProcBody__EmitAll+0x140            [m3front\src\misc\ProcBody.m3 @ 65]
 cm3!Module__CompileModule+0x2c1        [m3front\src\values\Module.m3 @ 1009]
 cm3!Module__Compile+0x3b5              [m3front\src\values\Module.m3 @ 940]
 cm3!M3Front__DoCompile+0xb61           [m3front\src\misc\M3Front.m3 @ 220]

(Unfortunately asserts in m3c are not useful, since m3cg_multipass runs though successfully recording all the state,
so then when m3c runs, you cannot see where m3front produced the bad data. This is the nature of iterating over long lived data structures instead of single pass w/o memory. But m3cg_multipass is good.)

I think this is an m3front bug, confusing byvalue and byref with structs or something.

 - Jay

________________________________
From: Jay K
Sent: Thursday, January 6, 2022 7:24 AM
To: m3devel <m3devel at elegosoft.com>
Subject: reference to variable exceeds its storage space

fyi:
C:\s\cm3\scripts\python\src\hash\MD5.m3(277) : warning C4739: reference to variable 'MD5_m_212_L_213' exceeds its storage space

    len64 := LOOPHOLE(VAL(ctrl.length + ctrl.bufLen,LONGINT) * 8L,Int64On32);

  Int64On32 = RECORD a, b: Int32;  END;

((*(INT64*)(&MD5_m_212_L_213))=(INT64)( ((INT64)(((INT64)( /*LH2*/ (INT64)((INT32)( ((INT32)(*((UINT32*)(4+((ADDRESS)(ctrl_L_114))))))+ ((INT32)(*((UINT32*)(ctrl_L_114))))))))*  INT64_(8)))));

 /* Var_Type2 */ ADDRESS MD5_m_212_L_213 = { 0 };

 /* Var_Type3 */ STRUCT(8) MD5_m_214_L_215 = { 0 };

..not sure yet if this is an m3c bug or m3front bug or what..not enough IR tracing in M3C..

Only a problem on 32bit targets.

I have local changes to m3c but I think this is an m3front bug. Investigating..

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/f6b368bd/attachment-0001.html>

From coder5506 at pobox.com  Thu Jan  6 20:52:24 2022
From: coder5506 at pobox.com (Eric Sessoms)
Date: Thu, 06 Jan 2022 14:52:24 -0500
Subject: [M3devel] making CI "optionally required"?
In-Reply-To: <MWHPR1401MB1951CF27553B69BD4F5A2CEFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>
 <MWHPR1401MB1951CF27553B69BD4F5A2CEFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <861bdc82-2c6b-45f6-a929-32df07fca480@www.fastmail.com>

Well, that was why I made a point to say it was on the settings page.  I've turned it off.

But it looks to me lke "nothing runs" because of the concurrency controls working as intended.  You have an old job cranking along, blocking the new jobs.  You can change that behavior if you want.

On Thu, Jan 6, 2022, at 2:13 PM, Jay K wrote:
> Um, now nothing runs?
>
>  - Jay
>
> *From:* Eric Sessoms <coder5506 at pobox.com>
> *Sent:* Thursday, January 6, 2022 6:09 PM
> *To:* Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
> *Subject:* Re: making CI "optionally required"? 
> 
> There's two steps.  First you have to enable "auto-merge" for the 
> repository.  I just did that, it's on the github settings page for CM3. 
>  Then you can set an individual PR to auto-merge.  (Supposedly. I 
> haven't used it.)
>
> See 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Fpull-requests%2Fcollaborating-with-pull-requests%2Fincorporating-changes-from-a-pull-request%2Fautomatically-merging-a-pull-request&amp;data=04%7C01%7C%7Cacb40d9a4bc64ec7cc3208d9d13fd009%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770894290626734%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=YT11VfKCziMOqQoZ5xl0ksAaggzuRMlRP%2B1H3cjUTrQ%3D&amp;reserved=0
>
> Given the way you've been working, you might want to play with the 
> concurrency options.  See 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fworkflow-syntax-for-github-actions%23concurrency&amp;data=04%7C01%7C%7Cacb40d9a4bc64ec7cc3208d9d13fd009%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770894290626734%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=JRt67aJINU7zSMKFgGjmDedxKyAQRLMH6V015ONCSOA%3D&amp;reserved=0
>
> Edit the file .github/workflows/integration.yml to change it.
>
> In particular, you might want to use the option to cancel an 
> in-progress run.  I recommend you don't remove the concurrency controls 
> altogether, because that's a quick way to get Github to block your 
> access to CI.
>
> For anything more fine-grained, I think you'd have to start editing the 
> workflow files.  For example, when I've wanted to run just one check 
> I've just commented out the ones I didn't want.  You can get fancier, 
> but then you're writing more code.
>
> On Thu, Jan 6, 2022, at 11:39 AM, Jay K wrote:
>> On a PR by PR basis, can I make checks required? Possibly check by 
>> check?
>> Like, "complete once checks pass", cause I am impatient and want to 
>> click it now, and have it wait/poll, not me.
>>
>>  - Jay

From jayk123 at hotmail.com  Thu Jan  6 20:53:43 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 19:53:43 +0000
Subject: [M3devel] making CI "optionally required"?
In-Reply-To: <861bdc82-2c6b-45f6-a929-32df07fca480@www.fastmail.com>
References: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>
 <MWHPR1401MB1951CF27553B69BD4F5A2CEFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <861bdc82-2c6b-45f6-a929-32df07fca480@www.fastmail.com>
Message-ID: <MWHPR1401MB1951F66761D2304DD982DF15E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

sorry, right, I kinda suspected that, I poked around looking for what is running.
If that is the case, then I will wait.

 - Jay

________________________________
From: Eric Sessoms <coder5506 at pobox.com>
Sent: Thursday, January 6, 2022 7:52 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: making CI "optionally required"?

Well, that was why I made a point to say it was on the settings page.  I've turned it off.

But it looks to me lke "nothing runs" because of the concurrency controls working as intended.  You have an old job cranking along, blocking the new jobs.  You can change that behavior if you want.

On Thu, Jan 6, 2022, at 2:13 PM, Jay K wrote:
> Um, now nothing runs?
>
>  - Jay
>
> *From:* Eric Sessoms <coder5506 at pobox.com>
> *Sent:* Thursday, January 6, 2022 6:09 PM
> *To:* Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
> *Subject:* Re: making CI "optionally required"?
>
> There's two steps.  First you have to enable "auto-merge" for the
> repository.  I just did that, it's on the github settings page for CM3.
>  Then you can set an individual PR to auto-merge.  (Supposedly. I
> haven't used it.)
>
> See
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Fpull-requests%2Fcollaborating-with-pull-requests%2Fincorporating-changes-from-a-pull-request%2Fautomatically-merging-a-pull-request&amp;data=04%7C01%7C%7C4442efcbcef146bd8b4708d9d14e1bad%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770955672587629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=WyENHo6a5zeQuHSJV%2FJnjrVUh9EShwvbdTn3w4l5lLI%3D&amp;reserved=0
>
> Given the way you've been working, you might want to play with the
> concurrency options.  See
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fworkflow-syntax-for-github-actions%23concurrency&amp;data=04%7C01%7C%7C4442efcbcef146bd8b4708d9d14e1bad%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770955672587629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=fnhyQ8yQc3jfhVIKI4Y5F2HYjYSRDPYfPEtyXERE154%3D&amp;reserved=0
>
> Edit the file .github/workflows/integration.yml to change it.
>
> In particular, you might want to use the option to cancel an
> in-progress run.  I recommend you don't remove the concurrency controls
> altogether, because that's a quick way to get Github to block your
> access to CI.
>
> For anything more fine-grained, I think you'd have to start editing the
> workflow files.  For example, when I've wanted to run just one check
> I've just commented out the ones I didn't want.  You can get fancier,
> but then you're writing more code.
>
> On Thu, Jan 6, 2022, at 11:39 AM, Jay K wrote:
>> On a PR by PR basis, can I make checks required? Possibly check by
>> check?
>> Like, "complete once checks pass", cause I am impatient and want to
>> click it now, and have it wait/poll, not me.
>>
>>  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/47b3bb7c/attachment.html>

From jayk123 at hotmail.com  Thu Jan  6 21:03:01 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 20:03:01 +0000
Subject: [M3devel] making CI "optionally required"?
In-Reply-To: <MWHPR1401MB1951F66761D2304DD982DF15E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>
 <MWHPR1401MB1951CF27553B69BD4F5A2CEFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <861bdc82-2c6b-45f6-a929-32df07fca480@www.fastmail.com>
 <MWHPR1401MB1951F66761D2304DD982DF15E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB19518ECD72C937E5FCB46ECFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

sorry to be dense here, but I canceled a running one, for a reason, and don't see how to run again.

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:53 PM
To: Eric Sessoms <coder5506 at pobox.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: making CI "optionally required"?

sorry, right, I kinda suspected that, I poked around looking for what is running.
If that is the case, then I will wait.

 - Jay

________________________________
From: Eric Sessoms <coder5506 at pobox.com>
Sent: Thursday, January 6, 2022 7:52 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: making CI "optionally required"?

Well, that was why I made a point to say it was on the settings page.  I've turned it off.

But it looks to me lke "nothing runs" because of the concurrency controls working as intended.  You have an old job cranking along, blocking the new jobs.  You can change that behavior if you want.

On Thu, Jan 6, 2022, at 2:13 PM, Jay K wrote:
> Um, now nothing runs?
>
>  - Jay
>
> *From:* Eric Sessoms <coder5506 at pobox.com>
> *Sent:* Thursday, January 6, 2022 6:09 PM
> *To:* Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
> *Subject:* Re: making CI "optionally required"?
>
> There's two steps.  First you have to enable "auto-merge" for the
> repository.  I just did that, it's on the github settings page for CM3.
>  Then you can set an individual PR to auto-merge.  (Supposedly. I
> haven't used it.)
>
> See
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Fpull-requests%2Fcollaborating-with-pull-requests%2Fincorporating-changes-from-a-pull-request%2Fautomatically-merging-a-pull-request&amp;data=04%7C01%7C%7C4442efcbcef146bd8b4708d9d14e1bad%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770955672587629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=WyENHo6a5zeQuHSJV%2FJnjrVUh9EShwvbdTn3w4l5lLI%3D&amp;reserved=0
>
> Given the way you've been working, you might want to play with the
> concurrency options.  See
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fworkflow-syntax-for-github-actions%23concurrency&amp;data=04%7C01%7C%7C4442efcbcef146bd8b4708d9d14e1bad%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770955672587629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=fnhyQ8yQc3jfhVIKI4Y5F2HYjYSRDPYfPEtyXERE154%3D&amp;reserved=0
>
> Edit the file .github/workflows/integration.yml to change it.
>
> In particular, you might want to use the option to cancel an
> in-progress run.  I recommend you don't remove the concurrency controls
> altogether, because that's a quick way to get Github to block your
> access to CI.
>
> For anything more fine-grained, I think you'd have to start editing the
> workflow files.  For example, when I've wanted to run just one check
> I've just commented out the ones I didn't want.  You can get fancier,
> but then you're writing more code.
>
> On Thu, Jan 6, 2022, at 11:39 AM, Jay K wrote:
>> On a PR by PR basis, can I make checks required? Possibly check by
>> check?
>> Like, "complete once checks pass", cause I am impatient and want to
>> click it now, and have it wait/poll, not me.
>>
>>  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/f68201f7/attachment-0001.html>

From jayk123 at hotmail.com  Thu Jan  6 21:05:46 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 6 Jan 2022 20:05:46 +0000
Subject: [M3devel] making CI "optionally required"?
In-Reply-To: <MWHPR1401MB19518ECD72C937E5FCB46ECFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951A04C4B3E9C6B4E43948AE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <99a49c3c-1343-4566-a2e0-9e4ec9ac9636@www.fastmail.com>
 <MWHPR1401MB1951CF27553B69BD4F5A2CEFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <861bdc82-2c6b-45f6-a929-32df07fca480@www.fastmail.com>
 <MWHPR1401MB1951F66761D2304DD982DF15E64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB19518ECD72C937E5FCB46ECFE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951D9D7CE93B2BE658BD34BE64C9@MWHPR1401MB1951.namprd14.prod.outlook.com>

hm. Sometimes there is "rerun all" and then one of those is waiting on "integration", of a PR I closed. Maybe I let it run anyway.

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 8:03 PM
To: Eric Sessoms <coder5506 at pobox.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] making CI "optionally required"?

sorry to be dense here, but I canceled a running one, for a reason, and don't see how to run again.

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 6, 2022 7:53 PM
To: Eric Sessoms <coder5506 at pobox.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: making CI "optionally required"?

sorry, right, I kinda suspected that, I poked around looking for what is running.
If that is the case, then I will wait.

 - Jay

________________________________
From: Eric Sessoms <coder5506 at pobox.com>
Sent: Thursday, January 6, 2022 7:52 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: making CI "optionally required"?

Well, that was why I made a point to say it was on the settings page.  I've turned it off.

But it looks to me lke "nothing runs" because of the concurrency controls working as intended.  You have an old job cranking along, blocking the new jobs.  You can change that behavior if you want.

On Thu, Jan 6, 2022, at 2:13 PM, Jay K wrote:
> Um, now nothing runs?
>
>  - Jay
>
> *From:* Eric Sessoms <coder5506 at pobox.com>
> *Sent:* Thursday, January 6, 2022 6:09 PM
> *To:* Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
> *Subject:* Re: making CI "optionally required"?
>
> There's two steps.  First you have to enable "auto-merge" for the
> repository.  I just did that, it's on the github settings page for CM3.
>  Then you can set an individual PR to auto-merge.  (Supposedly. I
> haven't used it.)
>
> See
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Fpull-requests%2Fcollaborating-with-pull-requests%2Fincorporating-changes-from-a-pull-request%2Fautomatically-merging-a-pull-request&amp;data=04%7C01%7C%7C4442efcbcef146bd8b4708d9d14e1bad%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770955672587629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=WyENHo6a5zeQuHSJV%2FJnjrVUh9EShwvbdTn3w4l5lLI%3D&amp;reserved=0<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Fpull-requests%2Fcollaborating-with-pull-requests%2Fincorporating-changes-from-a-pull-request%2Fautomatically-merging-a-pull-request&data=04%7C01%7C%7Cf740508c72ac49136e2e08d9d14f9109%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770961931478029%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BTxos5QTAu4p3YP9XUfCWtUr77mWVQ6fa2vYoMzeAls%3D&reserved=0>
>
> Given the way you've been working, you might want to play with the
> concurrency options.  See
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fworkflow-syntax-for-github-actions%23concurrency&amp;data=04%7C01%7C%7C4442efcbcef146bd8b4708d9d14e1bad%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770955672587629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=fnhyQ8yQc3jfhVIKI4Y5F2HYjYSRDPYfPEtyXERE154%3D&amp;reserved=0<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fworkflow-syntax-for-github-actions%23concurrency&data=04%7C01%7C%7Cf740508c72ac49136e2e08d9d14f9109%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637770961931478029%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JlgVQTRkQQR%2FIFmgQVLaL1A45eUd4XMwsdSHvpWkOwI%3D&reserved=0>
>
> Edit the file .github/workflows/integration.yml to change it.
>
> In particular, you might want to use the option to cancel an
> in-progress run.  I recommend you don't remove the concurrency controls
> altogether, because that's a quick way to get Github to block your
> access to CI.
>
> For anything more fine-grained, I think you'd have to start editing the
> workflow files.  For example, when I've wanted to run just one check
> I've just commented out the ones I didn't want.  You can get fancier,
> but then you're writing more code.
>
> On Thu, Jan 6, 2022, at 11:39 AM, Jay K wrote:
>> On a PR by PR basis, can I make checks required? Possibly check by
>> check?
>> Like, "complete once checks pass", cause I am impatient and want to
>> click it now, and have it wait/poll, not me.
>>
>>  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220106/e9305469/attachment.html>

From jayk123 at hotmail.com  Fri Jan  7 21:46:49 2022
From: jayk123 at hotmail.com (Jay K)
Date: Fri, 7 Jan 2022 20:46:49 +0000
Subject: [M3devel] m3core/Uerror
Message-ID: <MWHPR1401MB1951B583BE83F76DA679C33EE64D9@MWHPR1401MB1951.namprd14.prod.outlook.com>

libm3 has some errno functionality that turns out not to be portable.
It assumes errno is "small" and positive.
One or both of these is false, e.g. on Haiku and FreeBSD.
FreeBSD might not set errno to the high values, just define them.

I strongly suggest both assumptions be removed.

But it is a subtle change to a "public API".

1. Change CARDINAL to INTEGER. Not in the API actually, but in some users.
Uerror already uses int.

2. Redefine Uerror.Max's meaning. It is now just a meaningless
number, the same number as before, to keep some code compiling.
Alternatively, remove it, and let each user make up a number.

This will help Haiku and remove more porting cost to
hypothetical future operating systems. i.e. it will "just work" more often.

https://github.com/modula3/cm3/pull/842

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220107/255203fe/attachment-0001.html>

From jayk123 at hotmail.com  Sat Jan  8 09:10:50 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sat, 8 Jan 2022 08:10:50 +0000
Subject: [M3devel] dropping dynamic linking?
In-Reply-To: <SYBP282MB2755B02CF8900B351EB39E37B21E9@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB19513C63BE110E32AB353983E6079@MWHPR1401MB1951.namprd14.prod.outlook.com>,
 <SYBP282MB2755084EB677180271786F70B2079@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>,
 <MWHPR1401MB1951AF4F42DC69A2D7669C0FE61F9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB2755B02CF8900B351EB39E37B21E9@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB19515A3437D6BA46F3123301E64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Others had similar ideas:

https://cgit.freebsd.org/ports/log/net/cvsup?id=108df0eb691bcf9f242e5ef830bea1bb80db10d3&showmsg=1

Upgrade to snap-16.1f. This snapshot is primarily cleanups, John Polstra 2002-01-02 2 -32/+6

...

Remove the special static linking when PACKAGE_BUILDING is set.
When ezm3 is used, programs are always statically linked with the
Modula-3 libraries.  This means that binaries built with this port
will be stand-alone, but still dynamically linked with the system
libraries.  So, for example, they will work with runsocks

:)

 > I meant dynamic linking of libraries coded in Modula-3.

I find this still ambiguous, sorry.
Can we speak of "to" and not "of"?
What is being linked "to"?
to libc?
to m3core?

My goal is to stop worrying about:
 - maintaining the config files
 - worrying about running "non-installed" executables
 - which systems support $origin

Dynamic linking to libc would remain.
We don't do anything for that, it kinda just happens automatically. Kinda.
I realize there is flexibilty there.

Building m3core.so, and therefore linking to it, is kinda the bothersome point.
I do realize the use cases though, say of multiple Modula-3 codebases, multiple Modula-3 .so
files, loaded into the same process, and sharing one m3core.so, one garbage collector.

That is something that might not work well w/o dynamic linking.
Though in the ELF model, it might..the first one kinda wins. But that doesn't
work on Mac with the two level namespace.

Perhaps delegating to libtool is a solution though, I have been mulling as well.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Saturday, July 3, 2021 5:46 AM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: dropping dynamic linking?

I meant dynamic linking of libraries coded in Modula-3.


Get Outlook for iOS<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7C%7C6ba376bad4d84a3cfb7a08d93de5fabd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637608880234270278%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hVnl23ssW6Aj%2FkkEJ0DI4pPYEbXY8BhMbrU6Rm%2BSXOQ%3D&reserved=0>
________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Saturday, July 3, 2021 4:43:15 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: dropping dynamic linking?

Also, did you mean dynamic linking to Modula-3 i.e. m3core, or dynamically linked binaries in general, i.e. dynamically linked to libc?
These are really two separate things. Dynamic linking to libc is much more valuable and less negotiable and less problematic than dynamic linking to m3core. Some platforms have no public stable syscall interface, except via thin layer in libc, so dynamic linking is required. e.g. Darwin and Solaris I believe, and NT. Libc versioning/compatbility is also generally pretty good.

I was really only asking about m3core, etc., not libc.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Thursday, June 24, 2021 7:40 AM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: dropping dynamic linking?


Dropping dynamic linking is a non-starter. It must be supported!



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Thursday, June 24, 2021 at 2:22 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] dropping dynamic linking?

I am curious, if anyone would mind much, if we stopped doing dynamically linking, of Modula-3 code itself.



I am very well aware of all the tradeoffs.

Dynamic linking saves diskspace and memory when code is actually shared.

There is some indirection, but it should? not be much. But ELF defaults are also terrible.



The Modula-3 build system has a fair amount of code to support dynamic linking.

Is it worth maintaining?



We would continue dynamically linking to the underlying OS.



The Go developers seem to have a blanket policy/implementation of static linking (to go code, not OS).



 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220108/43af140e/attachment-0001.html>

From jayk123 at hotmail.com  Sat Jan  8 10:28:29 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sat, 8 Jan 2022 09:28:29 +0000
Subject: [M3devel] m3core/Uerror
In-Reply-To: <MWHPR1401MB1951B583BE83F76DA679C33EE64D9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951B583BE83F76DA679C33EE64D9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951A044818B80B28FB73A0DE64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

ok, furthermore..how about removing Uerror.Max?

There are three places in the cm3 tree that use it.
One in libm3 and two in the web browsers.
It can be replaced with an arbitrary integer at each use.

Or even the arrays replaced with something dynamically sized.
i.e. every Modula-3 program pays about 2k of address space and commit (about 250
pointers, 8 bytes each) for this array to cache the conversion of errno to a string. It is kinda wasteful.
(Granted..these days..I noticed Msvc2019 compiler and linker
taking on 128mb additional commit each just to start up, so 2k isn't much).
And really 1k is about right for Linux/Mac/Windows
(errno.h (apple.com)<https://opensource.apple.com/source/xnu/xnu-4570.71.2/bsd/sys/errno.h.auto.html> errno constants | Microsoft Docs<https://docs.microsoft.com/en-us/cpp/c-runtime-library/errno-constants?view=msvc-170>).



 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Friday, January 7, 2022 8:46 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] m3core/Uerror

libm3 has some errno functionality that turns out not to be portable.
It assumes errno is "small" and positive.
One or both of these is false, e.g. on Haiku and FreeBSD.
FreeBSD might not set errno to the high values, just define them.

I strongly suggest both assumptions be removed.

But it is a subtle change to a "public API".

1. Change CARDINAL to INTEGER. Not in the API actually, but in some users.
Uerror already uses int.

2. Redefine Uerror.Max's meaning. It is now just a meaningless
number, the same number as before, to keep some code compiling.
Alternatively, remove it, and let each user make up a number.

This will help Haiku and remove more porting cost to
hypothetical future operating systems. i.e. it will "just work" more often.

https://github.com/modula3/cm3/pull/842<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fpull%2F842&data=04%7C01%7C%7C7ced1e91a90b4318ecc208d9d21ed915%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637771852213983308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Q%2FUFmD07TAe9QCGu%2FCmV0pvo92gG%2F1WHVrtVCP4f4L4%3D&reserved=0>

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220108/e9209d6d/attachment.html>

From vvm at tut.by  Sat Jan  8 11:19:29 2022
From: vvm at tut.by (vvm at tut.by)
Date: Sat, 08 Jan 2022 13:19:29 +0300
Subject: [M3devel] dropping dynamic linking?
In-Reply-To: <MWHPR1401MB19515A3437D6BA46F3123301E64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB19513C63BE110E32AB353983E6079@MWHPR1401MB1951.namprd14.prod.outlook.com>,
 <SYBP282MB2755084EB677180271786F70B2079@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>,
 <MWHPR1401MB1951AF4F42DC69A2D7669C0FE61F9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB2755B02CF8900B351EB39E37B21E9@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB19515A3437D6BA46F3123301E64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <1606821641637052@mail.yandex.by>

An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220108/a2a236f1/attachment-0001.html>

From jayk123 at hotmail.com  Sat Jan  8 11:22:09 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sat, 8 Jan 2022 10:22:09 +0000
Subject: [M3devel] dropping dynamic linking?
In-Reply-To: <1606821641637052@mail.yandex.by>
References: <MWHPR1401MB19513C63BE110E32AB353983E6079@MWHPR1401MB1951.namprd14.prod.outlook.com>,
 <SYBP282MB2755084EB677180271786F70B2079@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>,
 <MWHPR1401MB1951AF4F42DC69A2D7669C0FE61F9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB2755B02CF8900B351EB39E37B21E9@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB19515A3437D6BA46F3123301E64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <1606821641637052@mail.yandex.by>
Message-ID: <MWHPR1401MB195108C05BB3DEA3B8F1869CE64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Yes. I'm also thinking about possible compatibility requirements if in a Linux distribution and an upgrade is done.
Like not having to worry about revising the version for changes.
Static libs and shared libs are both wonderful and horrible.

Also, I want to make an autotools target, that uses autoconf and automake, but I'm not sure I want to bring libtool in.

 - Jay

________________________________
From: vvm at tut.by <vvm at tut.by>
Sent: Saturday, January 8, 2022 10:19 AM
To: Jay K <jayk123 at hotmail.com>; Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] dropping dynamic linking?

Hi!


Q: Today AMD64_MINGW is static linked?


Best regards, Victor Miasnikov

08.01.2022, 11:11, "Jay K" <jayk123 at hotmail.com>:
Others had similar ideas:

https://cgit.freebsd.org/ports/log/net/cvsup?id=108df0eb691bcf9f242e5ef830bea1bb80db10d3&showmsg=1<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freebsd.org%2Fports%2Flog%2Fnet%2Fcvsup%3Fid%3D108df0eb691bcf9f242e5ef830bea1bb80db10d3%26showmsg%3D1&data=04%7C01%7C%7C48a1f1bcd906438437b308d9d2905c9c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637772339743489990%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gNwF1446t0l%2FQQGXAyc4Suo9s%2BEZqreD6iuGjxFGVlk%3D&reserved=0>

Upgrade to snap-16.1f. This snapshot is primarily cleanups, John Polstra 2002-01-02 2 -32/+6

...

Remove the special static linking when PACKAGE_BUILDING is set.
When ezm3 is used, programs are always statically linked with the
Modula-3 libraries.  This means that binaries built with this port
will be stand-alone, but still dynamically linked with the system
libraries.  So, for example, they will work with runsocks

:)

 > I meant dynamic linking of libraries coded in Modula-3.

I find this still ambiguous, sorry.
Can we speak of "to" and not "of"?
What is being linked "to"?
to libc?
to m3core?

My goal is to stop worrying about:
 - maintaining the config files
 - worrying about running "non-installed" executables
 - which systems support $origin

Dynamic linking to libc would remain.
We don't do anything for that, it kinda just happens automatically. Kinda.
I realize there is flexibilty there.

Building m3core.so, and therefore linking to it, is kinda the bothersome point.
I do realize the use cases though, say of multiple Modula-3 codebases, multiple Modula-3 .so
files, loaded into the same process, and sharing one m3core.so, one garbage collector.

That is something that might not work well w/o dynamic linking.
Though in the ELF model, it might..the first one kinda wins. But that doesn't
work on Mac with the two level namespace.

Perhaps delegating to libtool is a solution though, I have been mulling as well.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au<mailto:antony.hosking at anu.edu.au>>
Sent: Saturday, July 3, 2021 5:46 AM
To: Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>>; m3devel <m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>>
Subject: Re: dropping dynamic linking?

I meant dynamic linking of libraries coded in Modula-3.


Get Outlook for iOS<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7C%7C48a1f1bcd906438437b308d9d2905c9c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637772339743489990%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DSZP547XFyInrWafr%2Bf2KCRIVCrkfc40pYTiGUYzTX4%3D&reserved=0>
________________________________
From: Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>>
Sent: Saturday, July 3, 2021 4:43:15 AM
To: Tony Hosking <antony.hosking at anu.edu.au<mailto:antony.hosking at anu.edu.au>>; m3devel <m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>>
Subject: Re: dropping dynamic linking?

Also, did you mean dynamic linking to Modula-3 i.e. m3core, or dynamically linked binaries in general, i.e. dynamically linked to libc?
These are really two separate things. Dynamic linking to libc is much more valuable and less negotiable and less problematic than dynamic linking to m3core. Some platforms have no public stable syscall interface, except via thin layer in libc, so dynamic linking is required. e.g. Darwin and Solaris I believe, and NT. Libc versioning/compatbility is also generally pretty good.

I was really only asking about m3core, etc., not libc.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au<mailto:antony.hosking at anu.edu.au>>
Sent: Thursday, June 24, 2021 7:40 AM
To: Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>>; m3devel <m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>>
Subject: Re: dropping dynamic linking?


Dropping dynamic linking is a non-starter. It must be supported!



From: M3devel <m3devel-bounces at elegosoft.com<mailto:m3devel-bounces at elegosoft.com>> on behalf of Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>>
Date: Thursday, June 24, 2021 at 2:22 PM
To: m3devel <m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>>
Subject: [M3devel] dropping dynamic linking?

I am curious, if anyone would mind much, if we stopped doing dynamically linking, of Modula-3 code itself.



I am very well aware of all the tradeoffs.

Dynamic linking saves diskspace and memory when code is actually shared.

There is some indirection, but it should? not be much. But ELF defaults are also terrible.



The Modula-3 build system has a fair amount of code to support dynamic linking.

Is it worth maintaining?



We would continue dynamically linking to the underlying OS.



The Go developers seem to have a blanket policy/implementation of static linking (to go code, not OS).



 - Jay

,

_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C48a1f1bcd906438437b308d9d2905c9c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637772339743489990%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GAY%2BxpqzDGoqzuvvfXRGGITX%2Bb%2BZN80IqgQuGmXS63s%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220108/b7bcd9ba/attachment-0001.html>

From vvm at tut.by  Sat Jan  8 11:43:12 2022
From: vvm at tut.by (vvm at tut.by)
Date: Sat, 08 Jan 2022 13:43:12 +0300
Subject: [M3devel] dropping dynamic linking?
In-Reply-To: <MWHPR1401MB195108C05BB3DEA3B8F1869CE64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB19513C63BE110E32AB353983E6079@MWHPR1401MB1951.namprd14.prod.outlook.com>,
 <SYBP282MB2755084EB677180271786F70B2079@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>,
 <MWHPR1401MB1951AF4F42DC69A2D7669C0FE61F9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB2755B02CF8900B351EB39E37B21E9@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB19515A3437D6BA46F3123301E64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <1606821641637052@mail.yandex.by>
 <MWHPR1401MB195108C05BB3DEA3B8F1869CE64E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <317461641638201@mail.yandex.by>

An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220108/774f7a85/attachment.html>

From jayk123 at hotmail.com  Wed Jan 12 07:41:42 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 12 Jan 2022 06:41:42 +0000
Subject: [M3devel] Fw:  proc_waiter: error on read of child wait pipe 0x0,
 Win32 error 6
In-Reply-To: <MWHPR1401MB195104100F4D135F78351B86E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951673B53610E3D45F15235E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951AD1FD95142752195EF0DE6529@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB195104100F4D135F78351B86E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB195116B839D8D16A1F384E15E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>

wow, so much for m3_multithreaded_backend on Cygwin.

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Wednesday, January 12, 2022 6:27 AM
To: cygwin at sourceware.org <cygwin at sourceware.org>
Subject: Re: proc_waiter: error on read of child wait pipe 0x0, Win32 error 6

Ok, here is a small demonstration of the problem.

#include <stdlib.h>
#include <stdio.h>
#include <windows.h>

unsigned __stdcall thread(void* p)
{
  unsigned i;
  for (i = 0; i < 100; ++i)
   system("./a.exe");
  return 0;
}

int main()
{
unsigned i;
HANDLE threads[100] = {0};
FILE* f = fopen("a.c", "w");
fprintf(f, "int main() { return 0; }\n");
fclose(f);

system("g++ a.c");

for (i = 0; i < 100; ++i)
 threads[i] = CreateThread(0, 0, thread, 0,0,0);

for (i = 0; i < 100; ++i)
 WaitForSingleObject(threads[i], -1);

}

$ ./1.exe
      0 [main] sh 9287 C:\cygwin64\bin\sh.exe: *** fatal error - internal error reading the windows environment - too many environment variables?
                                                  3001 [main] sh 9287 cygwin_exception::open_stackdumpfile: Dumping stack trace to sh.exe.stackdump
                                                       0 [main] sh 9286 C:\cygwin64\bin\sh.exe: *** fatal error - internal error reading the windows environment - too many environment variables?
                                                                                                Stack trace:
Frame        Function    Args
000FFFFCD30  00180061C6E (001802795F0, 00180269E81, 00000000000, 000FFFFBB20)
000FFFFCD30  0018004849A (00000000228, 00800000160, 00000000000, 000FFFFCD30)
000FFFFCD30  001800484D2 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFCD30  0018005F12D (00180158C19, 00000000000, 00000000000, 000FFFFCD30)
000FFFFCD30  00180103AE4 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFCD30  00180048BC2 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFFFF0  00180047746 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFFFF0  001800477F4 (00000000000, 00000000000, 00000000000, 00000000000)
End of stack trace
                        0 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
             1078 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
     4452 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
                                                                                            sh: ./a.exe: Bad address
  11522 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
                                                                                          sh: ./a.exe: Bad address
sh: ./a.exe: Bad address
  38623 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
                                                                                            48009 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
                                                                                    48660 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
                                                                            49909 [waitproc] 1 9269 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6


I understand I can workaround this, in that the use of threads is silly.

Is this invalid, to mix Win32 threads with cygwin system()?

 - Jay

________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Wednesday, January 12, 2022 5:57 AM
To: cygwin at sourceware.org <cygwin at sourceware.org>
Subject: Re: proc_waiter: error on read of child wait pipe 0x0, Win32 error 6

Um, I know this isn't (yet) diligent/scientific, but must system create this thread?

#0  0x00007ff811e09e47 in KERNEL32!CreateThread () from /cygdrive/c/WINDOWS/System32/KERNEL32.DLL
#1  0x00000001800468eb in cygthread::create() () from /usr/bin/cygwin1.dll
#2  0x000000018010645f in pinfo::wait() () from /usr/bin/cygwin1.dll
#3  0x000000018012e562 in proc_subproc(unsigned int, unsigned long) () from /usr/bin/cygwin1.dll
#4  0x0000000180134e85 in child_info_spawn::worker(char const*, char const* const*, char const* const*, int, int, int) () from /usr/bin/cygwin1.dll
#5  0x0000000180136729 in spawnve () from /usr/bin/cygwin1.dll
#6  0x0000000180136d05 in spawnvp () from /usr/bin/cygwin1.dll
#7  0x0000000180144683 in system () from /usr/bin/cygwin1.dll
#8  0x000000018018effb in _sigfe () from /usr/bin/cygwin1.dll
#9  0x000000010052884b in QMachine__ExecCommand (t_L_563=0x448ef0 "\260\324\005",

'cause i.e. it complicates any lifetime/management.
I do understand some of the problems here, like needing separate threads to read stdout/stderr, if they are not equal.

 - Jay

________________________________
From: Jay K
Sent: Wednesday, January 12, 2022 4:12 AM
To: cygwin at sourceware.org <cygwin at sourceware.org>
Subject: proc_waiter: error on read of child wait pipe 0x0, Win32 error 6

I get this a lot:

      0 [waitproc] cm3 7641 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
    452 [waitproc] cm3 7641 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
    716 [waitproc] cm3 7641 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
   1652 [waitproc] cm3 7641 proc_waiter: error on read of child wait pipe 0x0, Win32 error 6
      0 [main] sh 7775 C:\cygwin64\bin\sh.exe: *** fatal error - internal error reading the windows environment - too many environment variables?

Scenario is multi-threaded "launching" a bunch of invocations of gcc.
"launching" was usually spawnve or fork/execve but I changed the code to use system and it still happens.
I might try to make a small reproducing, if this is not known.
It seems a bit difficult to believe I could cause this internal looking error.

Thank you,
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220112/5b0ac7cb/attachment-0001.html>

From jayk123 at hotmail.com  Wed Jan 12 09:01:13 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 12 Jan 2022 08:01:13 +0000
Subject: [M3devel] multithreaded backend inefficiencies and problems
Message-ID: <MWHPR1401MB1951CCBC3F3FCB923E2A6742E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>

Hi. While I grant that the multithreaded backend is much
better than single threaded, I think there is a "problem".


PROCEDURE ForceAllPromisesInParallel(promises : QPromiseSeq.T;
                                     parallelism : CARDINAL) =
  VAR
    curSeq  := NEW(QPromiseSeq.T).init();
    threads := NEW(RefSeq.T).init();
  BEGIN
    FOR i := 0 TO promises.size()-1 DO
      WITH p = promises.get(i) DO
        curSeq.addhi(p);
        IF i = promises.size()-1 OR ISTYPE(p,QPromise.Empty) THEN
          WITH cl = NEW(SeqClosure, seq := curSeq) DO
            threads.addhi (Thread.Fork(cl));

            IF threads.size() > parallelism-1 THEN
              EVAL Thread.Join(threads.remlo());
            END;

            curSeq := NEW(QPromiseSeq.T).init()
          END
        END
      END
    END;
    WHILE threads.size() > 0 DO EVAL Thread.Join(threads.remlo()) END;

    EVAL promises.init(); (* empty promises *)
  END ForceAllPromisesInParallel;

The problem is kinda the remlo calls.

What you really want is to wait for any of the processes
and as soon as any one terminates, run another.

So that if you queue up a few quick ones and a few long ones,
a small number of long ones don't starve out the initiation of
more quick ones.

In Win32 parlance we'd use WaitForMultipleObjects(any not all)
and as it returns, we'd run another.

In Unix parlance, waitpid can be wait for any process, but not
any of a specified set. (imagine this code is being used along
with other unrelated code in the process, that launches
and waits for processes)

In the Modula-3 code we create a thread per process.
This is unfortunate but I guess not too expensive, and can be leveraged.

I think something like a condition variable would work out.
I am somewhat trained to think in condition variables, and I think
Modula-3 approximates them.

Instead of waiting on any threads, we'd wait for the condition.
Each thread would wait on the one specific process it launched,
and any thread that gets a terminated process would signal the condition.

As well, this might lend itself to a thread pool, but that is a separate matter.

Another problem here is that Cygwin is buggy.
We kinda need to use spawn(nowait) and waitpid. But again there is
the problem of waiting for any of a set or a specific one...

I guess we can assign process group ids to wait on groups?
How does on specify them? Just make up our own numbers?
Use one of the process ids in the group as a process group id?

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220112/77545f95/attachment.html>

From jayk123 at hotmail.com  Wed Jan 12 14:31:02 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 12 Jan 2022 13:31:02 +0000
Subject: [M3devel] multithreaded backend inefficiencies and problems
In-Reply-To: <CAE62oi==rNPQV0+-xoz4zHT9ATAq_QxMn3sOny54qbJh9hgqvQ@mail.gmail.com>
References: <MWHPR1401MB1951CCBC3F3FCB923E2A6742E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oi==rNPQV0+-xoz4zHT9ATAq_QxMn3sOny54qbJh9hgqvQ@mail.gmail.com>
Message-ID: <MWHPR1401MB19519FE88E8076C0449D5009E6529@MWHPR1401MB1951.namprd14.prod.outlook.com>

Right, indeed it does work well and nothing broke.

That Cygwin does not work is definitely bugs in Cygwin.
For Cygwin I went ahead and have the autotools target, with make -j. It works.
I don't think a graph is super appropriate here, in that, within any one directory, I don't think there is much of a graph.
I mean, the graph is very simple and you can cheat.
All C++ compilation is independent of all other C++ compilation.
Both the C++ backend  and hand written C/C++.
A linking or making of libs, within one directory, is dependent on all C++ compilation in that directory.

Where it gets more interesting perhaps is that if we could build multiple directories concurrently, then, well, it is still simple.
Again all C++ compilation is independent of all other.
To the extent that we dynamically link, we could quickly build stub dlls with empty functions.
And then once the stub bottleneck is cleared, all linking can proceed in parallel as well.

A semaphore might be appropriate since this is about keeping the count limited.

Windows threads are a bit expensive to create. I think.
 But you still sped things up nicely.
However the model of a thread per process, instead of thread pool, I have seen. It does avoid some sychronization.
It is just that, you know, the threads are not doing anything, except run a process.
They are a waste.

 - Jay

________________________________
From: Mika Nystrom <mika.nystrom at gmail.com>
Sent: Wednesday, January 12, 2022 1:24 PM
To: Jay K <jayk123 at hotmail.com>
Cc: m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] multithreaded backend inefficiencies and problems

I think I am the responsible party here.  When I wrote this code, my focus was on getting a performance increase without Screwing Anything Up, so I deliberately chose a super-simple FIFO approach.

Modula-3 has condition variables that seem pretty normal to me, and the obvious approach to me would be to mark a boolean (one for each subprocess) and signal the condition variable when a subprocess exits.  The driver process would lock the mutex and just scan the booleans for the worker threads to find out which thread is done and replace it with the next task.  Really not that difficult.

Ah there are many other ways of doing this...

What would be even cooler would be some sort of graph structure keeping track of dependencies.

The overhead of spawning and collecting threads (at least on Linux) I think is really insignificant in this context.  I'd just put all the logic in the m3 threads and not mess with the system-level interactions, put those all in the worker thread for simplicity.

Not sure what to do about Cygwin.  Can its issues be isolated to the worker thread?

     Mika

On Wed, Jan 12, 2022 at 12:02 AM Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>> wrote:
Hi. While I grant that the multithreaded backend is much
better than single threaded, I think there is a "problem".


PROCEDURE ForceAllPromisesInParallel(promises : QPromiseSeq.T;
                                     parallelism : CARDINAL) =
  VAR
    curSeq  := NEW(QPromiseSeq.T).init();
    threads := NEW(RefSeq.T).init();
  BEGIN
    FOR i := 0 TO promises.size()-1 DO
      WITH p = promises.get(i) DO
        curSeq.addhi(p);
        IF i = promises.size()-1 OR ISTYPE(p,QPromise.Empty) THEN
          WITH cl = NEW(SeqClosure, seq := curSeq) DO
            threads.addhi (Thread.Fork(cl));

            IF threads.size() > parallelism-1 THEN
              EVAL Thread.Join(threads.remlo());
            END;

            curSeq := NEW(QPromiseSeq.T).init()
          END
        END
      END
    END;
    WHILE threads.size() > 0 DO EVAL Thread.Join(threads.remlo()) END;

    EVAL promises.init(); (* empty promises *)
  END ForceAllPromisesInParallel;

The problem is kinda the remlo calls.

What you really want is to wait for any of the processes
and as soon as any one terminates, run another.

So that if you queue up a few quick ones and a few long ones,
a small number of long ones don't starve out the initiation of
more quick ones.

In Win32 parlance we'd use WaitForMultipleObjects(any not all)
and as it returns, we'd run another.

In Unix parlance, waitpid can be wait for any process, but not
any of a specified set. (imagine this code is being used along
with other unrelated code in the process, that launches
and waits for processes)

In the Modula-3 code we create a thread per process.
This is unfortunate but I guess not too expensive, and can be leveraged.

I think something like a condition variable would work out.
I am somewhat trained to think in condition variables, and I think
Modula-3 approximates them.

Instead of waiting on any threads, we'd wait for the condition.
Each thread would wait on the one specific process it launched,
and any thread that gets a terminated process would signal the condition.

As well, this might lend itself to a thread pool, but that is a separate matter.

Another problem here is that Cygwin is buggy.
We kinda need to use spawn(nowait) and waitpid. But again there is
the problem of waiting for any of a set or a specific one...

I guess we can assign process group ids to wait on groups?
How does on specify them? Just make up our own numbers?
Use one of the process ids in the group as a process group id?

 - Jay
_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C7afafcbc93e74d20b94808d9d5cedf51%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637775906754524666%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FX%2FWjgn7MhKT09IhFVvcRiE4Lw49CA1bsasfBxtHMdQ%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220112/4a18915f/attachment-0001.html>

From jayk123 at hotmail.com  Thu Jan 13 09:47:48 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 13 Jan 2022 08:47:48 +0000
Subject: [M3devel] Fw: ExitProcess does not work in Cygwin?
In-Reply-To: <MWHPR1401MB1951896B7184CC9CC408CD07E6539@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951896B7184CC9CC408CD07E6539@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB195142A7ED6CC945ED9D76D5E6539@MWHPR1401MB1951.namprd14.prod.outlook.com>

fyi

I had Cygwin being a mix of Win32 and Posix, but that seem to find strange Cygwin behaviors.

 - Jay

________________________________
From: Jay K
Sent: Thursday, January 13, 2022 6:39 AM
To: cygwin at sourceware.org <cygwin at sourceware.org>
Subject: ExitProcess does not work in Cygwin?

ExitProcess does not work in Cygwin?

$ rm *.exe

# u is for Unix
# w is for Windows

$ cat u.c
#include <stdlib.h>
int main()
{
 exit(1);
}

$ gcc u.c
$ ./a.exe

$ echo $?
1

 => as expected

$ cat w.c
#include <windows.h>

int main()
{
 ExitProcess(1);
}

$ gcc w.c
$ ./a.exe

$ echo $?
0

 => not expected

$ uname -a
CYGWIN_NT-10.0 jayk-tp4 3.3.3(0.341/5/3) 2021-12-03 16:35 x86_64 Cygwin

works in debugger:

$ /cygdrive/c/bin/amd64/windbg.exe .\\a.exe

$ echo $?
1

?

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220113/91cd3591/attachment.html>

From jayk123 at hotmail.com  Fri Jan 14 02:10:46 2022
From: jayk123 at hotmail.com (Jay K)
Date: Fri, 14 Jan 2022 01:10:46 +0000
Subject: [M3devel] targets and file system structure
Message-ID: <MWHPR1401MB195174335F1B46458AFC6A9DE6549@MWHPR1401MB1951.namprd14.prod.outlook.com>

We have:

  /src/m3-libs/m3core/amd64_linux
  /install/pkg/m3core/amd64_linux

I propose roughly:
  /src/m3-libs/m3core/out
 or /out/m3core


and
 /install/pkg/m3core

or
  /amd64_linux/install/bin/cm3
  /amd64_linux/install/lib/libm3core.a
  /amd64_linux/install/lib/libm3core  # directory
    or /amd64_linux/install/pkgs/libm3core  # directory
  and the leading /amd64_linux is optional

In particular, the goal is to remove targets from the system entirely, both in intermediate directory structure and install structure.

I am not sure if pkg needs to be separate from lib.

I am not sure if a multi-arch pkg store is worth anything.
I think it was theoretically to assist cross builds.
But I don't like having a system that breaks ground in every single direction.

And I want porting to new systems to be almost a no-op.

I don't want M3_HOST. I don't want config files, except maybe a few lines
per target feeding into autoconf or cmake.
I'm willing for GET_PC to default 0 and feeling it in is optional.

I have driven way down what it takes to port, but not enough.

The install and inner loop experience should be:
 mkdir /out
 cd /out
 /src/configure -prefix=/foo (or cmake)
 make
 make install

Nothing should be written into the source tree.
DIrectory walk should be, not what it is today.
Driven either by automake subdirs, or cmake, or "recursive make is harmful".
 (I want both: flat makefile at root, and possibly cd around for manual partial builds, but maybe cd around is just make specific target?)

Or maybe my perspective is slightly off, and these are all separate
projects, already with read only source tree?

  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220114/7e179b5f/attachment-0001.html>

From vvm at tut.by  Fri Jan 14 08:53:53 2022
From: vvm at tut.by (vvm at tut.by)
Date: Fri, 14 Jan 2022 10:53:53 +0300
Subject: [M3devel] targets and file system structure
In-Reply-To: <MWHPR1401MB195174335F1B46458AFC6A9DE6549@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195174335F1B46458AFC6A9DE6549@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <515361642141517@mail.yandex.by>

An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220114/0e97c1d4/attachment.html>

From jayk123 at hotmail.com  Fri Jan 14 11:09:26 2022
From: jayk123 at hotmail.com (Jay K)
Date: Fri, 14 Jan 2022 10:09:26 +0000
Subject: [M3devel] targets and file system structure
In-Reply-To: <515361642141517@mail.yandex.by>
References: <MWHPR1401MB195174335F1B46458AFC6A9DE6549@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <515361642141517@mail.yandex.by>
Message-ID: <MWHPR1401MB1951A1787A6716EBB6F0A886E6549@MWHPR1401MB1951.namprd14.prod.outlook.com>

Right..I don't do a good job of using some sort of placeholder symbol vs. something more concrete.

/src is like $HOME/development/cm3
and /install is like /usr/local/cm3

however I have another vague proposal that /install should be viable to be just /usr/local
and we'd install to /usr/local/{bin,lib, maybe pkg}.

One thing I haven't looked into is what other not-C++ systems do, like Rust or Go that might have some kind of need for "pkg",
some other source required for building but not running.
I guess /usr/local/include sort of is a hint.

Maybe /usr/local/pkg/m3core/{*.m3x,*.ig} is the way.

Yes boot1 is a cross compile, that stops part way through.
A problem is every workflow needs to understood and accomodated, by every change.
Which is how the world is left with small changes.

In both Modula-3 and traditional C++ build systems, there is notion of two places.
pre-install and post-install

In Modula-3 /src/m3core/amd64_linux
and /install/pkg/m3core/amd64_linux and /install/lib

Once you get into dynamic linking and libtool you get this thing about if uninstalled binaries can run.

This bothers me, and is an area where maybe new-ish ground can be considered.
The idea that the original link might output the same structure as install, but to a different root, and using $ORIGIN.
Like make install DESTDIR= imho should not exist
and make dist should just be taring what make produced, without the make DESTDIR.
(or maybe make dist is sophisticated enough to take all paths to kinda cherry pick out of a larger structure?)

Mainly that executables should not be output by the linker in the same directory as .o files, and in the windows case also, pdbs should be output to a third directory.

Something like this:

mkdir /o/1
cd /o/1
/src/configure
make
make dist is tar xf /o/1/x
 again, skipping the make install DESTDIR kind of thing, and these binaries are directly runnable, even if dynamically linked, if $ORIGIN is available
make dist-symbols is tar xf /o/1/y
make install is cp -prv /o/1/x <prefix>, w/o need to pick out specific files
and all the .o files are in /o/1/z

Maybe I am too terse to make sense?
Maybe this is too much change to absorb or plan out incrementally?
Maybe dependency on ORIGIN is impractical?

  - Jay

________________________________
From: vvm at tut.by <vvm at tut.by>
Sent: Friday, January 14, 2022 7:53 AM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>; Eric Sessoms <coder5506 at pobox.com>
Subject: Re: [M3devel] targets and file system structure

Hi!


As I understand,  /src/  and  /install/ is "macros", i.e. pseudo-folders ?
/install/  is on practice  /usr/local/cm3/ ? And what is real folder of src-macro?


> /amd64_linux/install/lib/libm3core.a

What folder(s) is using for AMD64_NT ?
What are happened after run both
boot1.py AMD64_NT c
 and
boot1.py I386_NT
?

How about .pdb ?

> I am not sure if a multi-arch pkg store is worth anything.
> I think it was theoretically to assist cross builds.

As I see inside logs from
http://hudson.modula3.com:8080/job/cm3-current-test-all-pkgs-ZZZZZZZZZZ
as minimum Gygwin target ( a-ka NT386GNU ) was be cross-compiled.

In fact running "boot1.py I386_NT" on 64bit OS is crosscompiling too.


Best regards, Victor Miasnikov


14.01.2022, 04:14, "Jay K" <jayk123 at hotmail.com>:
We have:

  /src/m3-libs/m3core/amd64_linux
  /install/pkg/m3core/amd64_linux

I propose roughly:
  /src/m3-libs/m3core/out
 or /out/m3core


and
 /install/pkg/m3core

or
  /amd64_linux/install/bin/cm3
  /amd64_linux/install/lib/libm3core.a
  /amd64_linux/install/lib/libm3core  # directory
    or /amd64_linux/install/pkgs/libm3core  # directory
  and the leading /amd64_linux is optional

In particular, the goal is to remove targets from the system entirely, both in intermediate directory structure and install structure.

I am not sure if pkg needs to be separate from lib.

I am not sure if a multi-arch pkg store is worth anything.
I think it was theoretically to assist cross builds.
But I don't like having a system that breaks ground in every single direction.

And I want porting to new systems to be almost a no-op.

I don't want M3_HOST. I don't want config files, except maybe a few lines
per target feeding into autoconf or cmake.
I'm willing for GET_PC to default 0 and feeling it in is optional.

I have driven way down what it takes to port, but not enough.

The install and inner loop experience should be:
 mkdir /out
 cd /out
 /src/configure -prefix=/foo (or cmake)
 make
 make install

Nothing should be written into the source tree.
DIrectory walk should be, not what it is today.
Driven either by automake subdirs, or cmake, or "recursive make is harmful".
 (I want both: flat makefile at root, and possibly cd around for manual partial builds, but maybe cd around is just make specific target?)

Or maybe my perspective is slightly off, and these are all separate
projects, already with read only source tree?

  - Jay

,

_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C527839563791428d931a08d9d73303e3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637777436379758158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rgfV6RaL%2F19yakguH8ZZE1MaNDvVWI8l4IM5n9Rqs4I%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220114/05318d3a/attachment-0001.html>

From jayk123 at hotmail.com  Fri Jan 14 12:11:17 2022
From: jayk123 at hotmail.com (Jay K)
Date: Fri, 14 Jan 2022 11:11:17 +0000
Subject: [M3devel] targets and file system structure
In-Reply-To: <MWHPR1401MB1951A1787A6716EBB6F0A886E6549@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195174335F1B46458AFC6A9DE6549@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <515361642141517@mail.yandex.by>
 <MWHPR1401MB1951A1787A6716EBB6F0A886E6549@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951BD4D9AC439DB54E88CAEE6549@MWHPR1401MB1951.namprd14.prod.outlook.com>

Here is more of what I mean.

Think of autoconf, and its model of a "close enough Posix system".

Someone should be able to go Fuschia or DragonflyBSD and
do a build, without any edits to the source at all.

No M3_HOST, no GET_PC, no config file or at least no new config file.
 *Maybe* config file that says "posix, 64bit word, little endian",

"Just like hello world", seriously.

Anywhere a target is inserted into the middle or at end of path,
you should imagine instead it is at the start, placed by the user.

Or it is the output of config.guess, not anything in any hardcoded list.

We should have no hardcoded list of targets.

The sysroot model and the really out of source tree build model, including
out of a source tree containing multiple packages.
Kinda like how gcc builds, where you can put binutils into the tree.
Ideally even how gcc build manages build, host, and target.

I've been saying stuff like for years, but Eric's work automating
bootstrap, and just this week my autoconf work for Cygwin
gives kinda new push/belief that we can do it sooner than later.

Given a C++ backend and a Posix or Windows system, new systems should be zero work.

More so, but this is a ways out, given say WASI, not Posix, the difference is ideally
isolated to ifdef'ed or separate files of C or C++. So the generated .cpp files
remain portable across all systems (we don't yet have this for Posix vs. Win32, granted).

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Friday, January 14, 2022 10:09 AM
To: vvm at tut.by <vvm at tut.by>; m3devel <m3devel at elegosoft.com>; Eric Sessoms <coder5506 at pobox.com>
Subject: Re: [M3devel] targets and file system structure

Right..I don't do a good job of using some sort of placeholder symbol vs. something more concrete.

/src is like $HOME/development/cm3
and /install is like /usr/local/cm3

however I have another vague proposal that /install should be viable to be just /usr/local
and we'd install to /usr/local/{bin,lib, maybe pkg}.

One thing I haven't looked into is what other not-C++ systems do, like Rust or Go that might have some kind of need for "pkg",
some other source required for building but not running.
I guess /usr/local/include sort of is a hint.

Maybe /usr/local/pkg/m3core/{*.m3x,*.ig} is the way.

Yes boot1 is a cross compile, that stops part way through.
A problem is every workflow needs to understood and accomodated, by every change.
Which is how the world is left with small changes.

In both Modula-3 and traditional C++ build systems, there is notion of two places.
pre-install and post-install

In Modula-3 /src/m3core/amd64_linux
and /install/pkg/m3core/amd64_linux and /install/lib

Once you get into dynamic linking and libtool you get this thing about if uninstalled binaries can run.

This bothers me, and is an area where maybe new-ish ground can be considered.
The idea that the original link might output the same structure as install, but to a different root, and using $ORIGIN.
Like make install DESTDIR= imho should not exist
and make dist should just be taring what make produced, without the make DESTDIR.
(or maybe make dist is sophisticated enough to take all paths to kinda cherry pick out of a larger structure?)

Mainly that executables should not be output by the linker in the same directory as .o files, and in the windows case also, pdbs should be output to a third directory.

Something like this:

mkdir /o/1
cd /o/1
/src/configure
make
make dist is tar xf /o/1/x
 again, skipping the make install DESTDIR kind of thing, and these binaries are directly runnable, even if dynamically linked, if $ORIGIN is available
make dist-symbols is tar xf /o/1/y
make install is cp -prv /o/1/x <prefix>, w/o need to pick out specific files
and all the .o files are in /o/1/z

Maybe I am too terse to make sense?
Maybe this is too much change to absorb or plan out incrementally?
Maybe dependency on ORIGIN is impractical?

  - Jay

________________________________
From: vvm at tut.by <vvm at tut.by>
Sent: Friday, January 14, 2022 7:53 AM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>; Eric Sessoms <coder5506 at pobox.com>
Subject: Re: [M3devel] targets and file system structure

Hi!


As I understand,  /src/  and  /install/ is "macros", i.e. pseudo-folders ?
/install/  is on practice  /usr/local/cm3/ ? And what is real folder of src-macro?


> /amd64_linux/install/lib/libm3core.a

What folder(s) is using for AMD64_NT ?
What are happened after run both
boot1.py AMD64_NT c
 and
boot1.py I386_NT
?

How about .pdb ?

> I am not sure if a multi-arch pkg store is worth anything.
> I think it was theoretically to assist cross builds.

As I see inside logs from
http://hudson.modula3.com:8080/job/cm3-current-test-all-pkgs-ZZZZZZZZZZ
as minimum Gygwin target ( a-ka NT386GNU ) was be cross-compiled.

In fact running "boot1.py I386_NT" on 64bit OS is crosscompiling too.


Best regards, Victor Miasnikov


14.01.2022, 04:14, "Jay K" <jayk123 at hotmail.com>:
We have:

  /src/m3-libs/m3core/amd64_linux
  /install/pkg/m3core/amd64_linux

I propose roughly:
  /src/m3-libs/m3core/out
 or /out/m3core


and
 /install/pkg/m3core

or
  /amd64_linux/install/bin/cm3
  /amd64_linux/install/lib/libm3core.a
  /amd64_linux/install/lib/libm3core  # directory
    or /amd64_linux/install/pkgs/libm3core  # directory
  and the leading /amd64_linux is optional

In particular, the goal is to remove targets from the system entirely, both in intermediate directory structure and install structure.

I am not sure if pkg needs to be separate from lib.

I am not sure if a multi-arch pkg store is worth anything.
I think it was theoretically to assist cross builds.
But I don't like having a system that breaks ground in every single direction.

And I want porting to new systems to be almost a no-op.

I don't want M3_HOST. I don't want config files, except maybe a few lines
per target feeding into autoconf or cmake.
I'm willing for GET_PC to default 0 and feeling it in is optional.

I have driven way down what it takes to port, but not enough.

The install and inner loop experience should be:
 mkdir /out
 cd /out
 /src/configure -prefix=/foo (or cmake)
 make
 make install

Nothing should be written into the source tree.
DIrectory walk should be, not what it is today.
Driven either by automake subdirs, or cmake, or "recursive make is harmful".
 (I want both: flat makefile at root, and possibly cd around for manual partial builds, but maybe cd around is just make specific target?)

Or maybe my perspective is slightly off, and these are all separate
projects, already with read only source tree?

  - Jay

,

_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C6c8f55222c9b45e7d93408d9d74639b6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637777518887129446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=D6OmKXzmnu5vgKXRfTYSDWH3LbunG2LLZmXJJJzKXoU%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220114/b6fb3a8b/attachment-0001.html>

From jayk123 at hotmail.com  Sat Jan 15 11:03:16 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sat, 15 Jan 2022 10:03:16 +0000
Subject: [M3devel] theory of cooperative suspend?
Message-ID: <MWHPR1401MB1951B18DC9C5E555083CDDADE6559@MWHPR1401MB1951.namprd14.prod.outlook.com>

I'd like to understand cooperative suspend better.

In particular, when code is busy in Modula-3 code, I think I understand it.
You poll in about one of three ways:
 - conditional branch against global boolean; the simplest most portable way; this is what Mono does for example
  - you repeated call through a volatile global function pointer -- attack vector; I vaguely recall CoreCLR might do this?
  - you repeated read or write a page, which will either work or access violate/signal; this is what Java does for example

My choice would be the first, but that isn't my question.

My question is what do you do when calling out to "external" code?
Like long running kernel calls.
Or code that won't poll.

I'm actually confused how this is supposed to work even on Windows with preemptive suspend.

See the thing is, there are roughly 5 states:
 - busy in usermode in polling code
 - busy in usermode in non polling code
 - in an interruptible system service
  For example, I believe, maybe, SuspendThread + GetThreadContext on Windows can actually hang, or take a long time,
  like if the thread is in a large I/O. Or a long non-alertable wait.
  Or at the very least, you can go non-polling a long time.

So at the very least cooperatave suspend must at least entail a hybrid mode where it can be either by cooperative or preemptive.
Or rather, instead of preemptive, you can set a boolean, you kinda treat the thread as already suspended, and the boolean
is for when it returns, if the I/O complete, the thread will remain suspended.
And every exit from polling mode either must leave roots around, or SuspendThread + GetThreadContext is used.

I think on Unix the model might be that all system services or all long running system services are interruptable.
They will return with EINTR and handle any signals.

But also, merely touching memory, code, data, stack, can page in, can be long running. Individual instructions anywhere
can be considered to do I/O. I'm not sure what is to be done about that.

  - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220115/f30b9035/attachment.html>

From coder5506 at pobox.com  Sat Jan 15 20:37:04 2022
From: coder5506 at pobox.com (Eric Sessoms)
Date: Sat, 15 Jan 2022 14:37:04 -0500
Subject: [M3devel] theory of cooperative suspend?
In-Reply-To: <MWHPR1401MB1951B18DC9C5E555083CDDADE6559@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB1951B18DC9C5E555083CDDADE6559@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <9701ba89-72d5-40cb-9b72-e68a96e96c7b@www.fastmail.com>

I'm replying here less to provide an answer, and more to better understand the question.

On Sat, Jan 15, 2022, at 5:03 AM, Jay K wrote:
> My question is what do you do when calling out to "external" code?
> Like long running kernel calls.
> Or code that won't poll.

I must not be getting the question, because to me this describes the best-case scenario: you call the external service, block, and resume when it's done.  No problem.  If you want to keep the CPU busy, you just have a lot of threads.

Of course, this assumes your OS actually has a decent thread implementation.

> I'm actually confused how this is supposed to work even on Windows with 
> preemptive suspend.

...which of course Windows doesn't.  So then you want to multiplex user-level threads onto a sufficient pool of os-level threads, and use evented IO.  Libuv is a good choice for this b/c of the available libraries it works best on Windows, and Windows is where you need it.

(There's no particular advantage to evented IO on modern Unix systems, but it doesn't really hurt and it beats maintaining a second code base to deal with Windows.)

So then maybe, your question is "how"?  Afaik, there's nothing to it but a lot of grunt work: every potentionally blocking call has to get wrapped, at the lowest level possible, in code that suspends the user-thread and calls your chosen event api.  And you have to avoid making blocking calls in foreign code.

But since that still seems pretty basic, maybe your question is "how does one implement user-level threads"?  That's mostly just more grunt work (Tannenbaum is the usual reference), but this is the one place some theory does come into it: user-level threads can only be cooperative (you're not writing a vm).  You'll minimally need to yield before every blocking call, but additional yields can be helpful, as for GC.

I guess that's actually all the theory I'm aware of: preemptive is not an option, yields are required around blocking calls, but more yields can help... provided you have some practical use for them.

> Or rather, instead of preemptive, you can set a boolean, you kinda 
> treat the thread as already suspended, and the boolean

For user-level threads I think the "best" approach is lots of potential yield points in the generated code.  Mostly these potential yields will check a (os-) threadlocal cell looking for a request to actually yield.  You'll want to put them in the headers of non-optimized procedures, in the occassional loop iteration, of course in your IO wrappers, and anywhere that needs to interact with the storage manager.  (My definition of "best" is influenced by my interest in working on the GC, YMMV.)

> But also, merely touching memory, code, data, stack, can page in, can 
> be long running. Individual instructions anywhere
> can be considered to do I/O. I'm not sure what is to be done about that.

And that is hard.  The only approach I know to mitigate those issues is to be sure all OS-thread creation goes through an internal API, so that you have solid bookeeping and know what threads you have, where they are, and can find out what state they're in.  Anything that directly makes a kernel call (like FFI code) is going to cause problems.

This is probably all less helpful that I would hope, but if you refute all my suggestions I'll at least have a better idea of what you're trying to do.  :)

From jayk123 at hotmail.com  Mon Jan 17 22:31:26 2022
From: jayk123 at hotmail.com (Jay K)
Date: Mon, 17 Jan 2022 21:31:26 +0000
Subject: [M3devel] user threads use of self outside of critical?
Message-ID: <MWHPR1401MB195178D50B581572D3FF23D0E6579@MWHPR1401MB1951.namprd14.prod.outlook.com>

In the user thread code, how is it safe to use self outside of inCritical?

here:
PROCEDURE UnlockMutex (m: Mutex) =
  <*FATAL Alerted*>
  VAR waiters: BOOLEAN;
  BEGIN
    INC (inCritical);
      waiters := XRelease (m);
    DEC (inCritical);
    IF waiters AND inCritical = 0 THEN
      self.alertable := FALSE;
      InternalYield ();
    END;
  END UnlockMutex;


Because a mutex is held?

here:

PROCEDURE RunThread () =
  <*FATAL Alerted*>
  VAR xx: INTEGER;
  BEGIN
    self.stackbase := ADR(xx);
    handlerStack := self.handlers;
    Cerrno.SetErrno(self.errno);
    allow_sigvtalrm ();
    allow_othersigs ();
    DEC (inCritical);

    IF debug THEN
      DumpThread(self);
    END;
    FloatMode.InitThread (self.floatState);
    self.result := self.closure.apply ();

?

Thank you,
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220117/8472ccb9/attachment-0001.html>

From antony.hosking at anu.edu.au  Tue Jan 18 00:57:13 2022
From: antony.hosking at anu.edu.au (Tony Hosking)
Date: Mon, 17 Jan 2022 23:57:13 +0000
Subject: [M3devel] user threads use of self outside of critical?
In-Reply-To: <MWHPR1401MB195178D50B581572D3FF23D0E6579@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195178D50B581572D3FF23D0E6579@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <SYBP282MB275560733651E7E8F18E611AB2579@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>

In the user-threading code thread switches only occur at InternalYield (no preemption), so safe to use.

From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Tuesday, January 18, 2022 at 8:33 AM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] user threads use of self outside of critical?
In the user thread code, how is it safe to use self outside of inCritical?

here:
PROCEDURE UnlockMutex (m: Mutex) =
  <*FATAL Alerted*>
  VAR waiters: BOOLEAN;
  BEGIN
    INC (inCritical);
      waiters := XRelease (m);
    DEC (inCritical);
    IF waiters AND inCritical = 0 THEN
      self.alertable := FALSE;
      InternalYield ();
    END;
  END UnlockMutex;


Because a mutex is held?

here:

PROCEDURE RunThread () =
  <*FATAL Alerted*>
  VAR xx: INTEGER;
  BEGIN
    self.stackbase := ADR(xx);
    handlerStack := self.handlers;
    Cerrno.SetErrno(self.errno);
    allow_sigvtalrm ();
    allow_othersigs ();
    DEC (inCritical);

    IF debug THEN
      DumpThread(self);
    END;
    FloatMode.InitThread (self.floatState);
    self.result := self.closure.apply ();

?

Thank you,
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220117/6cd2026f/attachment.html>

From antony.hosking at anu.edu.au  Tue Jan 18 01:11:56 2022
From: antony.hosking at anu.edu.au (Tony Hosking)
Date: Tue, 18 Jan 2022 00:11:56 +0000
Subject: [M3devel] theory of cooperative suspend?
In-Reply-To: <9701ba89-72d5-40cb-9b72-e68a96e96c7b@www.fastmail.com>
References: <MWHPR1401MB1951B18DC9C5E555083CDDADE6559@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <9701ba89-72d5-40cb-9b72-e68a96e96c7b@www.fastmail.com>
Message-ID: <SYBP282MB27551E255E1BF8A6E1B8FA62B2579@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Eric Sessoms <coder5506 at pobox.com>
Date: Sunday, January 16, 2022 at 6:42 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] theory of cooperative suspend?
I'm replying here less to provide an answer, and more to better understand the question.

On Sat, Jan 15, 2022, at 5:03 AM, Jay K wrote:
> My question is what do you do when calling out to "external" code?
> Like long running kernel calls.
> Or code that won't poll.

I must not be getting the question, because to me this describes the best-case scenario: you call the external service, block, and resume when it's done.  No problem.  If you want to keep the CPU busy, you just have a lot of threads.

You can poll on entry/exit. Which means wrapping external calls.

Of course, this assumes your OS actually has a decent thread implementation.

> I'm actually confused how this is supposed to work even on Windows with
> preemptive suspend.

...which of course Windows doesn't.  So then you want to multiplex user-level threads onto a sufficient pool of os-level threads, and use evented IO.  Libuv is a good choice for this b/c of the available libraries it works best on Windows, and Windows is where you need it.

No need to multiplex. (Whether or not to multiplex depends on your view of what a Modula-3 thread should be. The systems programmer in me would like to think a Modula-3 thread is the same as an OS thread. But that may buy into overly heavyweight underlying OS threads. Multiplexing allows more lightweight language threads.) In any case, the compiler can generate the poll point before/after the external call. Assuming the external call does not manipulate the heap (can run safely in unmanaged mode), the thread can be marked as running external and so GC can occur whether it is blocked or active. You will still need a poll point on return to make sure the thread does not resume in managed mode without checking first that it is safe to do so. If not, the thread will need to either a) pause until notified by the GC (presumably in a stop-the-world phase where mutators are not supposed to be running), or b) synchronize with the GC (if the collector is concurrent or on-the-fly) to update stack roots or refresh GC state.



(There's no particular advantage to evented IO on modern Unix systems, but it doesn't really hurt and it beats maintaining a second code base to deal with Windows.)

So then maybe, your question is "how"?  Afaik, there's nothing to it but a lot of grunt work: every potentionally blocking call has to get wrapped, at the lowest level possible, in code that suspends the user-thread and calls your chosen event api.  And you have to avoid making blocking calls in foreign code.

But since that still seems pretty basic, maybe your question is "how does one implement user-level threads"?  That's mostly just more grunt work (Tannenbaum is the usual reference), but this is the one place some theory does come into it: user-level threads can only be cooperative (you're not writing a vm).  You'll minimally need to yield before every blocking call, but additional yields can be helpful, as for GC.

I guess that's actually all the theory I'm aware of: preemptive is not an option, yields are required around blocking calls, but more yields can help... provided you have some practical use for them.

> Or rather, instead of preemptive, you can set a boolean, you kinda
> treat the thread as already suspended, and the boolean

For user-level threads I think the "best" approach is lots of potential yield points in the generated code.  Mostly these potential yields will check a (os-) threadlocal cell looking for a request to actually yield.  You'll want to put them in the headers of non-optimized procedures, in the occassional loop iteration, of course in your IO wrappers, and anywhere that needs to interact with the storage manager.  (My definition of "best" is influenced by my interest in working on the GC, YMMV.)

> But also, merely touching memory, code, data, stack, can page in, can
> be long running. Individual instructions anywhere
> can be considered to do I/O. I'm not sure what is to be done about that.

And that is hard.  The only approach I know to mitigate those issues is to be sure all OS-thread creation goes through an internal API, so that you have solid bookeeping and know what threads you have, where they are, and can find out what state they're in.  Anything that directly makes a kernel call (like FFI code) is going to cause problems.

This is probably all less helpful that I would hope, but if you refute all my suggestions I'll at least have a better idea of what you're trying to do.  :)
_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&amp;data=04%7C01%7Cantony.hosking%40anu.edu.au%7C4fc38eae2e7a4904381f08d9d85f146b%7Ce37d725cab5c46249ae5f0533e486437%7C0%7C0%7C637778725573811852%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=Y%2B3r7hKlGx%2BEnRBjG3W%2BJ47%2FTur9p5%2FlxzY%2FHxrF9RE%3D&amp;reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220118/198d0041/attachment-0001.html>

From jayk123 at hotmail.com  Tue Jan 18 04:21:43 2022
From: jayk123 at hotmail.com (Jay K)
Date: Tue, 18 Jan 2022 03:21:43 +0000
Subject: [M3devel] theory of cooperative suspend?
In-Reply-To: <SYBP282MB27551E255E1BF8A6E1B8FA62B2579@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB1951B18DC9C5E555083CDDADE6559@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <9701ba89-72d5-40cb-9b72-e68a96e96c7b@www.fastmail.com>
 <SYBP282MB27551E255E1BF8A6E1B8FA62B2579@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB1951390747FDFE3020E778FDE6589@MWHPR1401MB1951.namprd14.prod.outlook.com>

My interest here is:

Removing preemptive suspend on "normal" Unix/NT systems.
It is a "bunch" of system-specific code, and not portable to some systems (e.g. WebAssembly?)
It is also wierd even on 32bit x86 on NT/amd64.

You know, part of the thing where the system has a stronger tendency to just work, no porting required.
You know, kinda next in line after the m3core/unix rewrite and C++ backend.
Consider for example the AppleWatch programming environment that has no setjmp/longjmp/make/get/swapcontext.

I don't really buy into user threads or multi-plexing.
But user threads are "needed" for MS-DOS (if MS-DOS is needed, unlikely).
 And for WebAssembly v1, but that is passing.

I realize thread creation is a bit expensive, etc. but it is difficult to compete with "one scheduler".
The best you can do, not easily, is write a scheduler w/o global knowledge, that does not work well,
unless there is nothing else being run by that kernel.

On the external call part, also have to deal with callbacks back to managed, right?
No big deal?
My mental model becomes not of call/return but just series of arbitrary jumps.
Because for example, you can "return" from kernel via I/O completion, and the program
has no obligation to return from that. It can be "continuation passing style".

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Tony Hosking <antony.hosking at anu.edu.au>
Sent: Tuesday, January 18, 2022 12:11 AM
To: Eric Sessoms <coder5506 at pobox.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] theory of cooperative suspend?






From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Eric Sessoms <coder5506 at pobox.com>
Date: Sunday, January 16, 2022 at 6:42 AM
To: m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] theory of cooperative suspend?

I'm replying here less to provide an answer, and more to better understand the question.

On Sat, Jan 15, 2022, at 5:03 AM, Jay K wrote:
> My question is what do you do when calling out to "external" code?
> Like long running kernel calls.
> Or code that won't poll.

I must not be getting the question, because to me this describes the best-case scenario: you call the external service, block, and resume when it's done.  No problem.  If you want to keep the CPU busy, you just have a lot of threads.



You can poll on entry/exit. Which means wrapping external calls.

Of course, this assumes your OS actually has a decent thread implementation.

> I'm actually confused how this is supposed to work even on Windows with
> preemptive suspend.

...which of course Windows doesn't.  So then you want to multiplex user-level threads onto a sufficient pool of os-level threads, and use evented IO.  Libuv is a good choice for this b/c of the available libraries it works best on Windows, and Windows is where you need it.



No need to multiplex. (Whether or not to multiplex depends on your view of what a Modula-3 thread should be. The systems programmer in me would like to think a Modula-3 thread is the same as an OS thread. But that may buy into overly heavyweight underlying OS threads. Multiplexing allows more lightweight language threads.) In any case, the compiler can generate the poll point before/after the external call. Assuming the external call does not manipulate the heap (can run safely in unmanaged mode), the thread can be marked as running external and so GC can occur whether it is blocked or active. You will still need a poll point on return to make sure the thread does not resume in managed mode without checking first that it is safe to do so. If not, the thread will need to either a) pause until notified by the GC (presumably in a stop-the-world phase where mutators are not supposed to be running), or b) synchronize with the GC (if the collector is concurrent or on-the-fly) to update stack roots or refresh GC state.




(There's no particular advantage to evented IO on modern Unix systems, but it doesn't really hurt and it beats maintaining a second code base to deal with Windows.)

So then maybe, your question is "how"?  Afaik, there's nothing to it but a lot of grunt work: every potentionally blocking call has to get wrapped, at the lowest level possible, in code that suspends the user-thread and calls your chosen event api.  And you have to avoid making blocking calls in foreign code.

But since that still seems pretty basic, maybe your question is "how does one implement user-level threads"?  That's mostly just more grunt work (Tannenbaum is the usual reference), but this is the one place some theory does come into it: user-level threads can only be cooperative (you're not writing a vm).  You'll minimally need to yield before every blocking call, but additional yields can be helpful, as for GC.

I guess that's actually all the theory I'm aware of: preemptive is not an option, yields are required around blocking calls, but more yields can help... provided you have some practical use for them.

> Or rather, instead of preemptive, you can set a boolean, you kinda
> treat the thread as already suspended, and the boolean

For user-level threads I think the "best" approach is lots of potential yield points in the generated code.  Mostly these potential yields will check a (os-) threadlocal cell looking for a request to actually yield.  You'll want to put them in the headers of non-optimized procedures, in the occassional loop iteration, of course in your IO wrappers, and anywhere that needs to interact with the storage manager.  (My definition of "best" is influenced by my interest in working on the GC, YMMV.)

> But also, merely touching memory, code, data, stack, can page in, can
> be long running. Individual instructions anywhere
> can be considered to do I/O. I'm not sure what is to be done about that.

And that is hard.  The only approach I know to mitigate those issues is to be sure all OS-thread creation goes through an internal API, so that you have solid bookeeping and know what threads you have, where they are, and can find out what state they're in.  Anything that directly makes a kernel call (like FFI code) is going to cause problems.

This is probably all less helpful that I would hope, but if you refute all my suggestions I'll at least have a better idea of what you're trying to do.  :)
_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&amp;data=04%7C01%7Cantony.hosking%40anu.edu.au%7C4fc38eae2e7a4904381f08d9d85f146b%7Ce37d725cab5c46249ae5f0533e486437%7C0%7C0%7C637778725573811852%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=Y%2B3r7hKlGx%2BEnRBjG3W%2BJ47%2FTur9p5%2FlxzY%2FHxrF9RE%3D&amp;reserved=0<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C96cb397dc1bf43d4de4b08d9da17792c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637780616631343466%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vd7HazzjywI1aeWKoMt5LTvqZCgJkRZdKW3XuyPR7Qg%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220118/dabc1674/attachment.html>

From jayk123 at hotmail.com  Tue Jan 18 04:25:59 2022
From: jayk123 at hotmail.com (Jay K)
Date: Tue, 18 Jan 2022 03:25:59 +0000
Subject: [M3devel] user threads use of self outside of critical?
In-Reply-To: <SYBP282MB275560733651E7E8F18E611AB2579@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB195178D50B581572D3FF23D0E6579@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275560733651E7E8F18E611AB2579@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB19518E1A323D586F5FB4EF4BE6589@MWHPR1401MB1951.namprd14.prod.outlook.com>

It is confusing because the timer interrupt can trigger fairly arbitrarily (when when blocked).
We really do need to Disable/EnableSwitching around all external calls, e.g. malloc, right?

I have this I386_DJGPP port building and starting up, but the context initialization isn't correct.
There is a working timer interrupt e.g. on FreeBSD but not DosBoxX.
Maybe I just need bite the proverbial bullet and poke around in the jmp_buf or write my own get/make/swapcontext, it isn't really difficult?
Or can it be done with setjmp/longjmp/alloca alone? I _kinda_ think it can.
i.e. if you alloca(alloca(1) - malloc()) roughly, in order to move stack pointer arbitrarily, which seems to work.
Maybe too gross and just write assembly? (There is all the assembly in the history, easy enough to dig out.).

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Monday, January 17, 2022 11:57 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: user threads use of self outside of critical?


In the user-threading code thread switches only occur at InternalYield (no preemption), so safe to use.



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Tuesday, January 18, 2022 at 8:33 AM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] user threads use of self outside of critical?

In the user thread code, how is it safe to use self outside of inCritical?



here:

PROCEDURE UnlockMutex (m: Mutex) =

  <*FATAL Alerted*>

  VAR waiters: BOOLEAN;

  BEGIN

    INC (inCritical);

      waiters := XRelease (m);

    DEC (inCritical);

    IF waiters AND inCritical = 0 THEN

      self.alertable := FALSE;

      InternalYield ();

    END;

  END UnlockMutex;





Because a mutex is held?



here:



PROCEDURE RunThread () =

  <*FATAL Alerted*>

  VAR xx: INTEGER;

  BEGIN

    self.stackbase := ADR(xx);

    handlerStack := self.handlers;

    Cerrno.SetErrno(self.errno);

    allow_sigvtalrm ();

    allow_othersigs ();

    DEC (inCritical);



    IF debug THEN

      DumpThread(self);

    END;

    FloatMode.InitThread (self.floatState);

    self.result := self.closure.apply ();



?



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220118/7d3339bb/attachment-0001.html>

From jayk123 at hotmail.com  Mon Jan 24 02:50:41 2022
From: jayk123 at hotmail.com (Jay K)
Date: Mon, 24 Jan 2022 01:50:41 +0000
Subject: [M3devel] zeroing locals?
Message-ID: <MWHPR1401MB195128AB22E094405618DB10E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Should backends zero locals upon function entry?

M3C is doing that.

There seems to be some dependency on it.
But maybe not much.

I haven't looked at the others.

There is a trickle of asking about performance, so I'd maybe like to remove that.
https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419
[https://opengraph.githubassets.com/784f31301f843ced166c8d52482f3c8b1123ad9e3bd0a432c75a0c5ccc731302/modula3/cm3/discussions/833]<https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419>
Compilation cm3-boot-unix64le-d5.11.1-20210610.cpp failed for AMD64 Haiku ? Discussion #833 ? modula3/cm3<https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419>
Hi, After a long period of activity in trying to compile CM3 for AMD64/Haiku, today I made another attempt. I managed to fix a lot of problems but there are two left and the first one is irremovabl...
github.com


i.e. here, the " = { 0 }" a few lines in.
https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108
[https://opengraph.githubassets.com/26ac2f3267578419038efd971829884a5eeafd1b35103dd3842093d9448e374b/modula3/cm3]<https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108>
cm3/M3C.m3 at a4acdd39bcb17252fe8a913d168b113716150649 ? modula3/cm3<https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108>
Critical Mass Modula-3. Contribute to modula3/cm3 development by creating an account on GitHub.
github.com



It is a murky area though, because uninitialized locals are the stuff of many bugs.

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220124/639fd081/attachment.html>

From jayk123 at hotmail.com  Mon Jan 24 08:47:20 2022
From: jayk123 at hotmail.com (Jay K)
Date: Mon, 24 Jan 2022 07:47:20 +0000
Subject: [M3devel] zeroing locals?
In-Reply-To: <MWHPR1401MB195128AB22E094405618DB10E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195128AB22E094405618DB10E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951B164E30339CA3B0D11CAE65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

I removed local initialization from m3c and looked into the failures some.

I see:

 - backend-generated temporaries
   Of course these are not m3front's problem, and m3c
   should continue to initialize them, roughly. It could probably be smarter.

 - structs; this is an area that imho
   has "never" worked "right", and will take longer to fix.
   I left m3c initializing them.
   I mean, I get that it is sometimes the right approach.
   It is problem of how much should m3front "lower" and what
   "level" a backend can "render" in. m3front lowers too much
   imho, for all backends, except the integrated one.

 - Sort of read before write, because of bitfields;
   M3C widens local bitfields to "round" integers and then
   reads them in their entirety in order to really
   only write and then read just part. This appears
   to the C++ compiler as using uninitialized data
   even though it is not.
   I put in a dumb analysis that sorta looks for read
   before write, and initializes.
   It should just for packed types and uninitialized
   but at first attempt that didn't work. Probably a problem
   of how m3c flows around typeinfo. Really m3c should be
   integrated into m3front and use its type system and type
   data directly, instead of all the hashes and declarations
   about them. And then e.g. not depend on hash uniqueness,
   which we have seen many problems with.

 - There might have been actually uninitialized locals
   from the Modula3 side, particularly in FileWin32.
   I have aggressively initialized in some Modula3 code.

- At least when using Visual C++ we should get errors
  if I missed anything. I realize it is tricky as to
  what code should analyze and error for this.

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Monday, January 24, 2022 1:50 AM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] zeroing locals?

Should backends zero locals upon function entry?

M3C is doing that.

There seems to be some dependency on it.
But maybe not much.

I haven't looked at the others.

There is a trickle of asking about performance, so I'd maybe like to remove that.
https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7Cc26a3be0b74640dfdffa08d9dedc4072%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637785859828710886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GU%2BR2V%2FYv625NhwPLLvRGxB02QSYOyz56DaEDHfMVZg%3D&reserved=0>
[https://opengraph.githubassets.com/784f31301f843ced166c8d52482f3c8b1123ad9e3bd0a432c75a0c5ccc731302/modula3/cm3/discussions/833]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7Cc26a3be0b74640dfdffa08d9dedc4072%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637785859828710886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GU%2BR2V%2FYv625NhwPLLvRGxB02QSYOyz56DaEDHfMVZg%3D&reserved=0>
Compilation cm3-boot-unix64le-d5.11.1-20210610.cpp failed for AMD64 Haiku ? Discussion #833 ? modula3/cm3<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7Cc26a3be0b74640dfdffa08d9dedc4072%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637785859828710886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GU%2BR2V%2FYv625NhwPLLvRGxB02QSYOyz56DaEDHfMVZg%3D&reserved=0>
Hi, After a long period of activity in trying to compile CM3 for AMD64/Haiku, today I made another attempt. I managed to fix a lot of problems but there are two left and the first one is irremovabl...
github.com


i.e. here, the " = { 0 }" a few lines in.
https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7Cc26a3be0b74640dfdffa08d9dedc4072%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637785859828710886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Iq9LBu9vnre%2FOMNAliJgHx7j%2BkVrrV8lqu4kA%2BnStEo%3D&reserved=0>
[https://opengraph.githubassets.com/26ac2f3267578419038efd971829884a5eeafd1b35103dd3842093d9448e374b/modula3/cm3]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7Cc26a3be0b74640dfdffa08d9dedc4072%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637785859828710886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Iq9LBu9vnre%2FOMNAliJgHx7j%2BkVrrV8lqu4kA%2BnStEo%3D&reserved=0>
cm3/M3C.m3 at a4acdd39bcb17252fe8a913d168b113716150649 ? modula3/cm3<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7Cc26a3be0b74640dfdffa08d9dedc4072%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637785859828710886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Iq9LBu9vnre%2FOMNAliJgHx7j%2BkVrrV8lqu4kA%2BnStEo%3D&reserved=0>
Critical Mass Modula-3. Contribute to modula3/cm3 development by creating an account on GitHub.
github.com



It is a murky area though, because uninitialized locals are the stuff of many bugs.

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220124/7b4fa787/attachment-0001.html>

From jayk123 at hotmail.com  Mon Jan 24 12:46:18 2022
From: jayk123 at hotmail.com (Jay K)
Date: Mon, 24 Jan 2022 11:46:18 +0000
Subject: [M3devel] zeroing locals?
In-Reply-To: <CAE62oimpBsWfta_nyVm8N_G0tXNV7ccK3L4f1rXVy2n17hL6dg@mail.gmail.com>
References: <MWHPR1401MB195128AB22E094405618DB10E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951B164E30339CA3B0D11CAE65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oimpBsWfta_nyVm8N_G0tXNV7ccK3L4f1rXVy2n17hL6dg@mail.gmail.com>
Message-ID: <MWHPR1401MB1951DB9386C750063F336D34E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

Yes, but I think, m3front handles this mostly..or entirely, really, depending on definition.

I should look more, but I didn't see the integrated backend handling it.
And things work pretty well without m3c handling it. Implying m3front is doing it.
The exceptions are, kinda understandable.
Even the struct part was probably ok, just beyond the analysis capability of Visual C++.
Ditto the bitfields, the code was ok, but Visual C++ is confused.

Also what I had in m3c, like regarding subranges, would not be wrong, just redundant.
I'd zero and then m3front would presumably do the subrange initialization.

I just really didn't look through much code, sorry.

I think these rules, this language definition, is a bit, um, out-dated.
.
What I mean is, I think they are super conventional for 20+ years ago, efficient, obvious.
But the world has mostly kinda moved on, and many languages take a kinda
more friendly reliable secure possibly slower position, either that there is zeroing of things,
or the language has to verify that there is no uninitialized data, possibly
via simplified algorithms, that do lead to sometimes excess initialization.
You know, computers are both faster and under more attack now.
Some not all of the old efficiencies matter less.

So I don't feel bad, if there is some consensus, that m3c or m3front
should aggressively zero.

Stack and even heap are becoming always zeroed:

Solving Uninitialized Stack Memory on Windows ? Microsoft Security Response Center<https://msrc-blog.microsoft.com/2020/05/13/solving-uninitialized-stack-memory-on-windows/>
ExAllocatePoolZero - Windows drivers | Microsoft Docs<https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-exallocatepoolzero>

Maybe we should be similar.

 - Jay

________________________________
From: Mika Nystrom <mika.nystrom at gmail.com>
Sent: Monday, January 24, 2022 11:19 AM
To: Jay K <jayk123 at hotmail.com>
Cc: m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] zeroing locals?

Does this have something to do with the language rule on variable initialization?

Per the Modula-3 report:

VAR id : T := E

declares id as a variable of type T whose initial value is the value of E.  Either ":= E" or ": T" can be omitted, but not both.  If T is omitted, it is taken to be the type of E.  If E is omitted, the initial value is an arbitrary value of type T.  If both are present, E must be assignable to T.

[emphasis mine]

It's important that reference types not point to random locations in memory.  So they are usually if not always initialized to NIL, which is usually (but not necessarily) a zero bit pattern.

Is there a dependence on the zeroing you're talking about to get this behavior?

Integer subranges have to be initialized to a value in the range by the same rule.  So VAR x : [1 .. 3] can't be initialized to zero.

     Mika

On Sun, Jan 23, 2022 at 11:48 PM Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>> wrote:
I removed local initialization from m3c and looked into the failures some.

I see:

 - backend-generated temporaries
   Of course these are not m3front's problem, and m3c
   should continue to initialize them, roughly. It could probably be smarter.

 - structs; this is an area that imho
   has "never" worked "right", and will take longer to fix.
   I left m3c initializing them.
   I mean, I get that it is sometimes the right approach.
   It is problem of how much should m3front "lower" and what
   "level" a backend can "render" in. m3front lowers too much
   imho, for all backends, except the integrated one.

 - Sort of read before write, because of bitfields;
   M3C widens local bitfields to "round" integers and then
   reads them in their entirety in order to really
   only write and then read just part. This appears
   to the C++ compiler as using uninitialized data
   even though it is not.
   I put in a dumb analysis that sorta looks for read
   before write, and initializes.
   It should just for packed types and uninitialized
   but at first attempt that didn't work. Probably a problem
   of how m3c flows around typeinfo. Really m3c should be
   integrated into m3front and use its type system and type
   data directly, instead of all the hashes and declarations
   about them. And then e.g. not depend on hash uniqueness,
   which we have seen many problems with.

 - There might have been actually uninitialized locals
   from the Modula3 side, particularly in FileWin32.
   I have aggressively initialized in some Modula3 code.

- At least when using Visual C++ we should get errors
  if I missed anything. I realize it is tricky as to
  what code should analyze and error for this.

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com<mailto:m3devel-bounces at elegosoft.com>> on behalf of Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>>
Sent: Monday, January 24, 2022 1:50 AM
To: m3devel <m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>>
Subject: [M3devel] zeroing locals?

Should backends zero locals upon function entry?

M3C is doing that.

There seems to be some dependency on it.
But maybe not much.

I haven't looked at the others.

There is a trickle of asking about performance, so I'd maybe like to remove that.
https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0tFBFc4KqXB8Z9MrJlHscGZ8xfFEw9J8Z02ftzQlY7k%3D&reserved=0>
[https://opengraph.githubassets.com/784f31301f843ced166c8d52482f3c8b1123ad9e3bd0a432c75a0c5ccc731302/modula3/cm3/discussions/833]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0tFBFc4KqXB8Z9MrJlHscGZ8xfFEw9J8Z02ftzQlY7k%3D&reserved=0>
Compilation cm3-boot-unix64le-d5.11.1-20210610.cpp failed for AMD64 Haiku ? Discussion #833 ? modula3/cm3<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0tFBFc4KqXB8Z9MrJlHscGZ8xfFEw9J8Z02ftzQlY7k%3D&reserved=0>
Hi, After a long period of activity in trying to compile CM3 for AMD64/Haiku, today I made another attempt. I managed to fix a lot of problems but there are two left and the first one is irremovabl...
github.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=961p8Puq3r3015qMelgVOFHQpJckCcrChXIczyN2Odc%3D&reserved=0>


i.e. here, the " = { 0 }" a few lines in.
https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ln6jXq8sw2x%2FEvQ0d5yNjkpUzdQzoNzFen2gDv4hnbc%3D&reserved=0>
[https://opengraph.githubassets.com/26ac2f3267578419038efd971829884a5eeafd1b35103dd3842093d9448e374b/modula3/cm3]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ln6jXq8sw2x%2FEvQ0d5yNjkpUzdQzoNzFen2gDv4hnbc%3D&reserved=0>
cm3/M3C.m3 at a4acdd39bcb17252fe8a913d168b113716150649 ? modula3/cm3<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ln6jXq8sw2x%2FEvQ0d5yNjkpUzdQzoNzFen2gDv4hnbc%3D&reserved=0>
Critical Mass Modula-3. Contribute to modula3/cm3 development by creating an account on GitHub.
github.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=961p8Puq3r3015qMelgVOFHQpJckCcrChXIczyN2Odc%3D&reserved=0>



It is a murky area though, because uninitialized locals are the stuff of many bugs.

 - Jay
_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4USspYtNbBAwKZJk5q0cl0NGxS0HlctiKPjuTezHbxM%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220124/0b19126a/attachment-0001.html>

From rodney.bates at gmx.com  Mon Jan 24 18:26:09 2022
From: rodney.bates at gmx.com (Rodney Bates)
Date: Mon, 24 Jan 2022 11:26:09 -0600
Subject: [M3devel] zeroing locals?
In-Reply-To: <MWHPR1401MB1951DB9386C750063F336D34E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195128AB22E094405618DB10E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951B164E30339CA3B0D11CAE65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oimpBsWfta_nyVm8N_G0tXNV7ccK3L4f1rXVy2n17hL6dg@mail.gmail.com>
 <MWHPR1401MB1951DB9386C750063F336D34E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <47d14f26-b97f-2ccc-c4ec-8c55bdd4ade3@gmx.com>



On 1/24/22 5:46 AM, Jay K wrote:
> Yes, but I think, m3front handles this mostly..or entirely, really, depending on definition.
>
> I should look more, but I didn't see the integrated backend handling it.
> And things work pretty well without m3c handling it. Implying m3front is doing it.

I believe the front end correctly implements the language.? More specifically,
it generates IR to initialize either to the source-coded initial constant,
if there is one, or its (the front end's) choice of a bit pattern that is the
low-level representation of some member of the high-level type. This in all
cases, not just locals.

It does a bit of optimization by initializing certain entire blocks of variables
to binary zeros, when that either satisfies the above for all variables in the
block, or maybe it selectively then overwrites a few variables in the block.

I don't recall ever altering any of this, but have vetted it a good bit, and am
not aware of any bugs.? In any case, I think it really is the front end's job,
and back ends need only translate the IR operations they receive. Of course,
a back end must take care of its own back-end-generated temporaries.? Presumably,
it would always generate write-before-read anyway.

Warnings are for the benefit of human coders.? When code is machine-generated,
the goal should be that it is correct, thus any warnings from the C++ compiler whould
be for things the generator has already ensured are harmless. Yes, the warnings can
be confusing/annoying and it's nice to try to avoid them.? Except when doing
development work on m3front/M3C, dare I suggest just using C++ compiler flags
to suppress them?


> The exceptions are, kinda understandable.
> Even the struct part was probably ok, just beyond the analysis capability of Visual C++.
> Ditto the bitfields, the code was ok, but Visual C++ is confused.
>
> Also what I had in m3c, like regarding subranges, would not be wrong, just redundant.
> I'd zero and then m3front would presumably do the subrange initialization.

Yes.
>
> I just really didn't look through much code, sorry.

Don't apologize.? That's not a trivial task.
>
> I think these rules, this language definition, is a bit, um, out-dated.
> .
> What I mean is, I think they are super conventional for 20+ years ago, efficient, obvious.
> But the world has mostly kinda moved on, and many languages take a kinda
> more friendly reliable secure possibly slower position, either that there is zeroing of things,
> or the language has to verify that there is no uninitialized data, possibly
> via simplified algorithms, that do lead to sometimes excess initialization.
> You know, computers are both faster and under more attack now.
> Some not all of the old efficiencies matter less.

Modula3's rule is the minimum necessary to keep the type system in non-UNSAFE
code watertight, leaving what latitude is possible for efficiency. Personally,
I have long thought that, even 20 years or more ago, it would have been
better to define exactly what default initial values were for all types.
But that might have pushed more important things out of the 50-page limit.

>
> So I don't feel bad, if there is some consensus, that m3c or m3front
> should aggressively zero.

Yes.? Avoiding redundant initializations in today's world is way down
in the performance noise.
>
> Stack and even heap are becoming always zeroed:
>
> Solving Uninitialized Stack Memory on Windows ? Microsoft Security Response Center <https://msrc-blog.microsoft.com/2020/05/13/solving-uninitialized-stack-memory-on-windows/>
> ExAllocatePoolZero - Windows drivers | Microsoft Docs <https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-exallocatepoolzero>
>
> Maybe we should be similar.
>
> ?- Jay
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> *From:* Mika Nystrom <mika.nystrom at gmail.com>
> *Sent:* Monday, January 24, 2022 11:19 AM
> *To:* Jay K <jayk123 at hotmail.com>
> *Cc:* m3devel <m3devel at elegosoft.com>
> *Subject:* Re: [M3devel] zeroing locals?
> Does this have something to do with the language rule on variable initialization?
>
> Per the Modula-3 report:
>
> VAR id : T := E
>
> declares id as a variable of type T whose initial value is the value of E.? Either ":= E" or ": T" can be omitted, but not both.? If T is omitted, it is taken to be the type of E. /If E is omitted, the initial value is an arbitrary value of type T. / If both are present, E must be assignable to T.
>
> [emphasis mine]
>
> It's important that reference types not point to random locations in memory.? So they are usually if not always initialized to NIL, which is usually (but not necessarily) a zero bit pattern.
>
> Is there a dependence on the zeroing you're talking about to get this behavior?
>
> Integer subranges have to be initialized to a value in the range by the same rule.? So VAR x : [1 .. 3] can't be initialized to zero.
>
> ? ? ?Mika
>
> On Sun, Jan 23, 2022 at 11:48 PM Jay K <jayk123 at hotmail.com <mailto:jayk123 at hotmail.com>> wrote:
>
>     I removed local initialization from m3c and looked into the failures some.
>
>     I see:
>
>     ?- backend-generated temporaries
>     ? ?Of course these are not m3front's problem, and m3c
>     ? ?should continue to initialize them, roughly. It could probably be smarter.
>
>     ?- structs; this is an area that imho
>     ? ?has "never" worked "right", and will take longer to fix.
>     ? ?I left m3c initializing them.
>     ? ?I mean, I get that it is sometimes the right approach.
>     ? ?It is problem of how much should m3front "lower" and what
>     ? ?"level" a backend can "render" in. m3front lowers too much
>     ? ?imho, for all backends, except the integrated one.
>
>     ?- Sort of read before write, because of bitfields;
>     ? ?M3C widens local bitfields to "round" integers and then
>     ? ?reads them in their entirety in order to really
>     ? ?only write and then read just part. This appears
>     ? ?to the C++ compiler as using uninitialized data
>     ? ?even though it is not.
>     ? ?I put in a dumb analysis that sorta looks for read
>     ? ?before write, and initializes.
>     ? ?It should just for packed types and uninitialized
>     ? ?but at first attempt that didn't work. Probably a problem
>     ? ?of how m3c flows around typeinfo. Really m3c should be
>     ? ?integrated into m3front and use its type system and type
>     ? ?data directly, instead of all the hashes and declarations
>     ? ?about them. And then e.g. not depend on hash uniqueness,
>     ? ?which we have seen many problems with.
>
>     ?- There might have been actually uninitialized locals
>     ? ?from the Modula3 side, particularly in FileWin32.
>     ? ?I have aggressively initialized in some Modula3 code.
>
>     - At least when using Visual C++ we should get errors
>     ? if I missed anything. I realize it is tricky as to
>     ? what code should analyze and error for this.
>
>     ?- Jay
>
>     ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>     *From:* M3devel <m3devel-bounces at elegosoft.com <mailto:m3devel-bounces at elegosoft.com>> on behalf of Jay K <jayk123 at hotmail.com <mailto:jayk123 at hotmail.com>>
>     *Sent:* Monday, January 24, 2022 1:50 AM
>     *To:* m3devel <m3devel at elegosoft.com <mailto:m3devel at elegosoft.com>>
>     *Subject:* [M3devel] zeroing locals?
>     Should backends zero locals upon function entry?
>
>     M3C is doing that.
>
>     There seems to be some dependency on it.
>     But maybe not much.
>
>     I haven't looked at the others.
>
>     There is a trickle of asking about performance, so I'd maybe like to remove that.
>     https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0tFBFc4KqXB8Z9MrJlHscGZ8xfFEw9J8Z02ftzQlY7k%3D&reserved=0>
>     <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0tFBFc4KqXB8Z9MrJlHscGZ8xfFEw9J8Z02ftzQlY7k%3D&reserved=0>
>     	
>     Compilation cm3-boot-unix64le-d5.11.1-20210610.cpp failed for AMD64 Haiku ? Discussion #833 ? modula3/cm3 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0tFBFc4KqXB8Z9MrJlHscGZ8xfFEw9J8Z02ftzQlY7k%3D&reserved=0>
>     Hi, After a long period of activity in trying to compile CM3 for AMD64/Haiku, today I made another attempt. I managed to fix a lot of problems but there are two left and the first one is irremovabl...
>     github.com <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=961p8Puq3r3015qMelgVOFHQpJckCcrChXIczyN2Odc%3D&reserved=0>
>
>
>
>     i.e. here, the "?= { 0 }" a few lines in.
>     https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ln6jXq8sw2x%2FEvQ0d5yNjkpUzdQzoNzFen2gDv4hnbc%3D&reserved=0>
>     <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ln6jXq8sw2x%2FEvQ0d5yNjkpUzdQzoNzFen2gDv4hnbc%3D&reserved=0>
>     	
>     cm3/M3C.m3 at a4acdd39bcb17252fe8a913d168b113716150649 ? modula3/cm3 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ln6jXq8sw2x%2FEvQ0d5yNjkpUzdQzoNzFen2gDv4hnbc%3D&reserved=0>
>     Critical Mass Modula-3. Contribute to modula3/cm3 development by creating an account on GitHub.
>     github.com <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=961p8Puq3r3015qMelgVOFHQpJckCcrChXIczyN2Odc%3D&reserved=0>
>
>
>
>
>     It is a murky area though, because uninitialized locals are the stuff of many bugs.
>
>     ?- Jay
>     _______________________________________________
>     M3devel mailing list
>     M3devel at elegosoft.com <mailto:M3devel at elegosoft.com>
>     https://m3lists.elegosoft.com/mailman/listinfo/m3devel <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5d0e91e7e350440ae93508d9df2b6d36%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786199873533391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4USspYtNbBAwKZJk5q0cl0NGxS0HlctiKPjuTezHbxM%3D&reserved=0>
>
>
> _______________________________________________
> M3devel mailing list
> M3devel at elegosoft.com
> https://m3lists.elegosoft.com/mailman/listinfo/m3devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220124/80a8ebef/attachment-0001.html>

From jayk123 at hotmail.com  Mon Jan 24 18:41:15 2022
From: jayk123 at hotmail.com (Jay K)
Date: Mon, 24 Jan 2022 17:41:15 +0000
Subject: [M3devel] zeroing locals?
In-Reply-To: <47d14f26-b97f-2ccc-c4ec-8c55bdd4ade3@gmx.com>
References: <MWHPR1401MB195128AB22E094405618DB10E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951B164E30339CA3B0D11CAE65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <CAE62oimpBsWfta_nyVm8N_G0tXNV7ccK3L4f1rXVy2n17hL6dg@mail.gmail.com>
 <MWHPR1401MB1951DB9386C750063F336D34E65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <47d14f26-b97f-2ccc-c4ec-8c55bdd4ade3@gmx.com>
Message-ID: <MWHPR1401MB1951DEC35B094D7057F6E2CAE65E9@MWHPR1401MB1951.namprd14.prod.outlook.com>

> But that might have pushed more important things out of the 50-page limit.

Btw, I tend to agree, this was the minimal wording, and makes
for an easier implementation btw.

Very simple seeming things, turn out to be very complicated,
in the specification and implementation of other languages.

Just having unsigned types and signed types, and letting them mix,
is not just complicated, but really quite unsolvable and broken.
There are various rules about how they combine in C and C++
and even with all the rules, it doesn't turn out sensible.
It isn't that some smart folks handled it for others.

I think we can go past 50 though.

 > I believe the front end correctly implements the language.

While this wording is tautological, I see it as a bit gray,
as to the divivion of labor between the various parts.
The compiler implements the language. The frontend is not obligated
to do it all. The IR is really much too low level for most backends.

 > dare I suggest just using C++ compiler flags to suppress them?

So yeah, I kinda know. There is this thing, that cfront was
criticized as not being a "real compiler". Stroustrup defended
it, roughly, it understood the entire language and any errors
from the C compiler were cfront bugs. m3front/m3c are kinda in the same
position. But I am willing to believe there are m3front bugs and am
reluctant to turn off the detection. It has been largely useful.
And targeting multiple compilers is also a nuisance.
You do end up relishing the assembler as a most lenient backend.

 > Presumably, it would always generate write-before-read anyway.

This is gray..I'm not sure what the frontend issues for bitfield locals.
i.e. I'm not sure where the rounding up of 20 bits to 32bits occurs.

I kinda suspect it is worthwhile to have the backend pass bit sizes not byte sizes.
Or both. But then again, it passes the typeuid, so passing any size is redundant.
And yet again, for not-m3cc, it should just pass m3front.Type.T, not do all this
proxying beyond a typeuid and passing data about it. Seriously.
We end up two+ parallel program representations.
I think I did an ok/good thing writing m3c at the m3cg level in the first place.
Leaving m3front alone, but now that it works well, and some of the rough parts
of m3cg realized, I think it should live above the m3cg level, or at least kinda
fuse them. i.e. pass the in-proc Type.T, for other backends to ignore.
But LLVM would likely benefit too.
Or move all the writing to be custom code in m3front, no pluggable interface.

 > Yes.  Avoiding redundant initializations in today's world
 > is way down in the performance noise.

Problem is I have no numbers and there are general performance concerns.
But also C++ compiler can optimize much away, if we asked.


 - Jay

________________________________
From: Rodney Bates <rodney.bates at gmx.com>
Sent: Monday, January 24, 2022 5:26 PM
To: Jay K <jayk123 at hotmail.com>; Mika Nystrom <mika.nystrom at gmail.com>
Cc: m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] zeroing locals?



On 1/24/22 5:46 AM, Jay K wrote:
Yes, but I think, m3front handles this mostly..or entirely, really, depending on definition.

I should look more, but I didn't see the integrated backend handling it.
And things work pretty well without m3c handling it. Implying m3front is doing it.

I believe the front end correctly implements the language.  More specifically,
it generates IR to initialize either to the source-coded initial constant,
if there is one, or its (the front end's) choice of a bit pattern that is the
low-level representation of some member of the high-level type. This in all
cases, not just locals.

It does a bit of optimization by initializing certain entire blocks of variables
to binary zeros, when that either satisfies the above for all variables in the
block, or maybe it selectively then overwrites a few variables in the block.

I don't recall ever altering any of this, but have vetted it a good bit, and am
not aware of any bugs.  In any case, I think it really is the front end's job,
and back ends need only translate the IR operations they receive.  Of course,
a back end must take care of its own back-end-generated temporaries.  Presumably,
it would always generate write-before-read anyway.

Warnings are for the benefit of human coders.  When code is machine-generated,
the goal should be that it is correct, thus any warnings from the C++ compiler whould
be for things the generator has already ensured are harmless.  Yes, the warnings can
be confusing/annoying and it's nice to try to avoid them.  Except when doing
development work on m3front/M3C, dare I suggest just using C++ compiler flags
to suppress them?


The exceptions are, kinda understandable.
Even the struct part was probably ok, just beyond the analysis capability of Visual C++.
Ditto the bitfields, the code was ok, but Visual C++ is confused.

Also what I had in m3c, like regarding subranges, would not be wrong, just redundant.
I'd zero and then m3front would presumably do the subrange initialization.

Yes.

I just really didn't look through much code, sorry.

Don't apologize.  That's not a trivial task.

I think these rules, this language definition, is a bit, um, out-dated.
.
What I mean is, I think they are super conventional for 20+ years ago, efficient, obvious.
But the world has mostly kinda moved on, and many languages take a kinda
more friendly reliable secure possibly slower position, either that there is zeroing of things,
or the language has to verify that there is no uninitialized data, possibly
via simplified algorithms, that do lead to sometimes excess initialization.
You know, computers are both faster and under more attack now.
Some not all of the old efficiencies matter less.

Modula3's rule is the minimum necessary to keep the type system in non-UNSAFE
code watertight, leaving what latitude is possible for efficiency. Personally,
I have long thought that, even 20 years or more ago, it would have been
better to define exactly what default initial values were for all types.
But that might have pushed more important things out of the 50-page limit.


So I don't feel bad, if there is some consensus, that m3c or m3front
should aggressively zero.

Yes.  Avoiding redundant initializations in today's world is way down
in the performance noise.

Stack and even heap are becoming always zeroed:

Solving Uninitialized Stack Memory on Windows ? Microsoft Security Response Center<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmsrc-blog.microsoft.com%2F2020%2F05%2F13%2Fsolving-uninitialized-stack-memory-on-windows%2F&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=S3B%2FnwkSldwS52a8aa3ml8m01VL1j9pf2Gr%2BBS5y9X0%3D&reserved=0>
ExAllocatePoolZero - Windows drivers | Microsoft Docs<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fwindows-hardware%2Fdrivers%2Fddi%2Fwdm%2Fnf-wdm-exallocatepoolzero&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=H%2Bz7c65Qd5OA%2F8O3O4z25ScacgzHSVzhUKY%2FMpVkGpw%3D&reserved=0>

Maybe we should be similar.

 - Jay

________________________________
From: Mika Nystrom <mika.nystrom at gmail.com><mailto:mika.nystrom at gmail.com>
Sent: Monday, January 24, 2022 11:19 AM
To: Jay K <jayk123 at hotmail.com><mailto:jayk123 at hotmail.com>
Cc: m3devel <m3devel at elegosoft.com><mailto:m3devel at elegosoft.com>
Subject: Re: [M3devel] zeroing locals?

Does this have something to do with the language rule on variable initialization?

Per the Modula-3 report:

VAR id : T := E

declares id as a variable of type T whose initial value is the value of E.  Either ":= E" or ": T" can be omitted, but not both.  If T is omitted, it is taken to be the type of E.  If E is omitted, the initial value is an arbitrary value of type T.  If both are present, E must be assignable to T.

[emphasis mine]

It's important that reference types not point to random locations in memory.  So they are usually if not always initialized to NIL, which is usually (but not necessarily) a zero bit pattern.

Is there a dependence on the zeroing you're talking about to get this behavior?

Integer subranges have to be initialized to a value in the range by the same rule.  So VAR x : [1 .. 3] can't be initialized to zero.

     Mika

On Sun, Jan 23, 2022 at 11:48 PM Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>> wrote:
I removed local initialization from m3c and looked into the failures some.

I see:

 - backend-generated temporaries
   Of course these are not m3front's problem, and m3c
   should continue to initialize them, roughly. It could probably be smarter.

 - structs; this is an area that imho
   has "never" worked "right", and will take longer to fix.
   I left m3c initializing them.
   I mean, I get that it is sometimes the right approach.
   It is problem of how much should m3front "lower" and what
   "level" a backend can "render" in. m3front lowers too much
   imho, for all backends, except the integrated one.

 - Sort of read before write, because of bitfields;
   M3C widens local bitfields to "round" integers and then
   reads them in their entirety in order to really
   only write and then read just part. This appears
   to the C++ compiler as using uninitialized data
   even though it is not.
   I put in a dumb analysis that sorta looks for read
   before write, and initializes.
   It should just for packed types and uninitialized
   but at first attempt that didn't work. Probably a problem
   of how m3c flows around typeinfo. Really m3c should be
   integrated into m3front and use its type system and type
   data directly, instead of all the hashes and declarations
   about them. And then e.g. not depend on hash uniqueness,
   which we have seen many problems with.

 - There might have been actually uninitialized locals
   from the Modula3 side, particularly in FileWin32.
   I have aggressively initialized in some Modula3 code.

- At least when using Visual C++ we should get errors
  if I missed anything. I realize it is tricky as to
  what code should analyze and error for this.

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com<mailto:m3devel-bounces at elegosoft.com>> on behalf of Jay K <jayk123 at hotmail.com<mailto:jayk123 at hotmail.com>>
Sent: Monday, January 24, 2022 1:50 AM
To: m3devel <m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>>
Subject: [M3devel] zeroing locals?

Should backends zero locals upon function entry?

M3C is doing that.

There seems to be some dependency on it.
But maybe not much.

I haven't looked at the others.

There is a trickle of asking about performance, so I'd maybe like to remove that.
https://github.com/modula3/cm3/discussions/833#discussioncomment-2022419<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=f7biVvMi3e6Qdl0GxUaV5fwFFKXVIhZjO1dSVse8008%3D&reserved=0>
[https://opengraph.githubassets.com/784f31301f843ced166c8d52482f3c8b1123ad9e3bd0a432c75a0c5ccc731302/modula3/cm3/discussions/833]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=f7biVvMi3e6Qdl0GxUaV5fwFFKXVIhZjO1dSVse8008%3D&reserved=0>
Compilation cm3-boot-unix64le-d5.11.1-20210610.cpp failed for AMD64 Haiku ? Discussion #833 ? modula3/cm3<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fdiscussions%2F833%23discussioncomment-2022419&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=f7biVvMi3e6Qdl0GxUaV5fwFFKXVIhZjO1dSVse8008%3D&reserved=0>
Hi, After a long period of activity in trying to compile CM3 for AMD64/Haiku, today I made another attempt. I managed to fix a lot of problems but there are two left and the first one is irremovabl...
github.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kTNfxBzSpdQ2%2BYzum9EYurLvXL6tt8jahB6XO7GuIyI%3D&reserved=0>


i.e. here, the " = { 0 }" a few lines in.
https://github.com/modula3/cm3/blob/a4acdd39bcb17252fe8a913d168b113716150649/m3-sys/m3back/src/M3C.m3#L6108<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2F8o%2BxWZzV5FTSv%2B72JYuXDZqMZyXmdBKpZm%2BfQQkBA8%3D&reserved=0>
[https://opengraph.githubassets.com/26ac2f3267578419038efd971829884a5eeafd1b35103dd3842093d9448e374b/modula3/cm3]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2F8o%2BxWZzV5FTSv%2B72JYuXDZqMZyXmdBKpZm%2BfQQkBA8%3D&reserved=0>
cm3/M3C.m3 at a4acdd39bcb17252fe8a913d168b113716150649 ? modula3/cm3<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fblob%2Fa4acdd39bcb17252fe8a913d168b113716150649%2Fm3-sys%2Fm3back%2Fsrc%2FM3C.m3%23L6108&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2F8o%2BxWZzV5FTSv%2B72JYuXDZqMZyXmdBKpZm%2BfQQkBA8%3D&reserved=0>
Critical Mass Modula-3. Contribute to modula3/cm3 development by creating an account on GitHub.
github.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843557299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kTNfxBzSpdQ2%2BYzum9EYurLvXL6tt8jahB6XO7GuIyI%3D&reserved=0>



It is a murky area though, because uninitialized locals are the stuff of many bugs.

 - Jay
_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843713544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zmDdKlEgHdsMwNu%2FdfDsb7G4EZ47P2YbKlpS%2BhvhD1k%3D&reserved=0>



_______________________________________________
M3devel mailing list
M3devel at elegosoft.com<mailto:M3devel at elegosoft.com>
https://m3lists.elegosoft.com/mailman/listinfo/m3devel<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C3f4a52359a7544235d7308d9df5ea3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637786419843713544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zmDdKlEgHdsMwNu%2FdfDsb7G4EZ47P2YbKlpS%2BhvhD1k%3D&reserved=0>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220124/f72133a8/attachment-0001.html>

From jayk123 at hotmail.com  Wed Jan 26 13:15:20 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 26 Jan 2022 12:15:20 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
Message-ID: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>

cm3 is running kinda on MS-DOS. Decades late, sorry. :)
I think I have user threads working even.

In DosBos-X it can generate all the cpp files for m3core.
And then fails, because, autoconf not found.
I can either fix the autotools path, or run compiler more directly.


Under 32bit Windows 10 it doesn't get quite as far:
***    <*ASSERT*> failed: NOT h.forwarded
***    file "../src/runtime/common/RTCollector.m3", line 599
***

PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =
  BEGIN
    WHILE h < he DO
      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>
      <* ASSERT NOT h.forwarded *>
      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN
        IF r # PromoteReason.OldImpure OR h.dirty THEN
          h.dirty := FALSE;
          h.gray := TRUE;
        END;
      END;
      INC(h, ADRSIZE(Header) + ReferentSize(h));
    END;
  END GrayBetween;

I think I saw similar, on IA64_HPUX, but, that was when I had
the system as little endian, whereas IA64_HPUX is big endian.

That is not likely the problem here, especially since it works under DosBox-X.

Any ideas?

Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.

It is possible the DosBox-X environment has more memory and the garbage collector
does not work there either.

Thank you,
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220126/74cc6ab6/attachment.html>

From antony.hosking at anu.edu.au  Wed Jan 26 14:14:00 2022
From: antony.hosking at anu.edu.au (Tony Hosking)
Date: Wed, 26 Jan 2022 13:14:00 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>

That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?

From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
cm3 is running kinda on MS-DOS. Decades late, sorry. :)
I think I have user threads working even.

In DosBos-X it can generate all the cpp files for m3core.
And then fails, because, autoconf not found.
I can either fix the autotools path, or run compiler more directly.


Under 32bit Windows 10 it doesn't get quite as far:
***    <*ASSERT*> failed: NOT h.forwarded
***    file "../src/runtime/common/RTCollector.m3", line 599
***

PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =
  BEGIN
    WHILE h < he DO
      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>
      <* ASSERT NOT h.forwarded *>
      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN
        IF r # PromoteReason.OldImpure OR h.dirty THEN
          h.dirty := FALSE;
          h.gray := TRUE;
        END;
      END;
      INC(h, ADRSIZE(Header) + ReferentSize(h));
    END;
  END GrayBetween;

I think I saw similar, on IA64_HPUX, but, that was when I had
the system as little endian, whereas IA64_HPUX is big endian.

That is not likely the problem here, especially since it works under DosBox-X.

Any ideas?

Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.

It is possible the DosBox-X environment has more memory and the garbage collector
does not work there either.

Thank you,
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220126/0c2c2c15/attachment-0001.html>

From jayk123 at hotmail.com  Wed Jan 26 14:21:01 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 26 Jan 2022 13:21:01 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>

Not exactly: MS-DOS, in 32bit Windows.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220126/0ad2d32a/attachment.html>

From antony.hosking at anu.edu.au  Wed Jan 26 14:22:06 2022
From: antony.hosking at anu.edu.au (Tony Hosking)
Date: Wed, 26 Jan 2022 13:22:06 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>

Aha, interesting. Any more clues?

From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.
Not exactly: MS-DOS, in 32bit Windows.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220126/86243606/attachment-0001.html>

From jayk123 at hotmail.com  Wed Jan 26 18:18:06 2022
From: jayk123 at hotmail.com (Jay K)
Date: Wed, 26 Jan 2022 17:18:06 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>

Not much. It is a kinda harsh environment.
Sometimes gdb works, sometimes not.
 (On win10-32 I can break on main, but then gdb fails to resume.)
Sometimes long file names work, sometimes not.
 (I don't try to succeed w/o them.)
Sometimes command lines are restricted to 126 characters, sometimes not.
 (Djgpp is supposed to workaround this, but it does not always work, again, on Win10-32.)
Something like "gcc -c foo.c" exceeds 126 characters because of the underlying cc1/ld.

I will try to try with Win9x and/or FreeDOS or such.
 If it works on more than one maybe declare success.
I don't think any non-Windows MS-DOS from Microsoft has long file names
and I am reluctant to fight that..we'd probably need to start transparently
read/writing .zips or something in m3core.

One unusual thing here is that I am using use threads,
and a customization thereof. Swapping context in timer interrupt.
 https://github.com/modula3/cm3/pull/896/files
but as long as it working in some environments I am satisfied.

I will try to ensure GC is running in DosBox (printf of write(1) about it),
and/or limit memory to induce pressure.

There are interesting things we can do, with more work,
since we are a compiler. For example the generated code
could maintain an easy to walk stack, and print it for assertion failures.

We could even implement a sort of debugger, where we make a call
after every line, to what normally just returns, but can pause
for input, and such.

But I'm not sure how much (more) I am going to put into an MS-DOS port. :)

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:22 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


Aha, interesting. Any more clues?



From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not exactly: MS-DOS, in 32bit Windows.



 - Jay



________________________________

From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.



That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220126/96ab1cac/attachment-0001.html>

From antony.hosking at anu.edu.au  Wed Jan 26 22:18:11 2022
From: antony.hosking at anu.edu.au (Tony Hosking)
Date: Wed, 26 Jan 2022 21:18:11 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>

Indeed, MS-DOS is just a curiosity at this stage.


Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 27, 2022 4:18:06 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not much. It is a kinda harsh environment.
Sometimes gdb works, sometimes not.
 (On win10-32 I can break on main, but then gdb fails to resume.)
Sometimes long file names work, sometimes not.
 (I don't try to succeed w/o them.)
Sometimes command lines are restricted to 126 characters, sometimes not.
 (Djgpp is supposed to workaround this, but it does not always work, again, on Win10-32.)
Something like "gcc -c foo.c" exceeds 126 characters because of the underlying cc1/ld.

I will try to try with Win9x and/or FreeDOS or such.
 If it works on more than one maybe declare success.
I don't think any non-Windows MS-DOS from Microsoft has long file names
and I am reluctant to fight that..we'd probably need to start transparently
read/writing .zips or something in m3core.

One unusual thing here is that I am using use threads,
and a customization thereof. Swapping context in timer interrupt.
 https://github.com/modula3/cm3/pull/896/files
but as long as it working in some environments I am satisfied.

I will try to ensure GC is running in DosBox (printf of write(1) about it),
and/or limit memory to induce pressure.

There are interesting things we can do, with more work,
since we are a compiler. For example the generated code
could maintain an easy to walk stack, and print it for assertion failures.

We could even implement a sort of debugger, where we make a call
after every line, to what normally just returns, but can pause
for input, and such.

But I'm not sure how much (more) I am going to put into an MS-DOS port. :)

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:22 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


Aha, interesting. Any more clues?



From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not exactly: MS-DOS, in 32bit Windows.



 - Jay



________________________________

From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.



That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220126/31ba8096/attachment-0001.html>

From jayk123 at hotmail.com  Thu Jan 27 14:48:52 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 27 Jan 2022 13:48:52 +0000
Subject: [M3devel] Pathname.LastBase win32 bug
Message-ID: <MWHPR1401MB195173775D73B2943417E3ADE6219@MWHPR1401MB1951.namprd14.prod.outlook.com>

Oh wow..we have

Pathname.LastBase

The intent imho is:

Pathname.LastBase("a.b.c/d.e.f") => d.e
split on last slash and then last period.

I'm not sure why you would ever want not-last Base, like:

 not-last Base(a.b/c) => a

but that is besides the point.

Anyway, we have Posix and Win32 forms.
The code is very similar. A copy/paste mess, that should not be written that way.
The factoring should be finer grained, not forking the entire file.

In both cases most of the work is procedure NameSections.

With these inscrutable names "Ubp" and "Lbp". Huh?

Posix:

PROCEDURE NameSections
.
.
.
        ELSIF ch = ExtSepChar THEN
          IF NOT extSepSeen THEN
            baseUpb := pos; extSepSeen := TRUE
          END
        END
.
.
.
  END NameSections;


Win32:


PROCEDURE NameSections
.
.
.
        ELSIF ch = ExtSepChar THEN
          baseUpb := pos
        END
.
.
.

As a result we have:

 Posix(a.b.c/d.e.f) => d.e correct
 Win32(a.b.c/d.e.f) => d incorrect

 I noticed, because DJGPP uses Win32 paths -- colon and either slash and case insensitive,
 and many of my gcc and g++ crash, so I'm running cc1plus and as directly,
 forming paths, and wasn't getting what I expected.
 Foo.m3.cpp should have been turning into Foo.m3 but was turning into Foo.

So I guess the DOS port ended up useful. ?

(It appears well on its way toward building itself, albeit quite slowly, is in libm3, I had to use Win98SE DOS (others would likely work), with long name support.)

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220127/2f33db48/attachment.html>

From jayk123 at hotmail.com  Thu Jan 27 23:23:07 2022
From: jayk123 at hotmail.com (Jay K)
Date: Thu, 27 Jan 2022 22:23:07 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB19511E6B1F48CED1BD1FD2BDE6219@MWHPR1401MB1951.namprd14.prod.outlook.com>

Well, using Win98 MS-DOS (not booting into Windows, that didn't work), I was able to do one pass self-build.
 i.e. m3core libm3 m3objfile m3linker cm3 m3front set sysutils m3back etc.

The Python scripts didn't quite work, so I resorted to bash, alas.
I didn't step though or do extra printing but I expect garbage collection and user threads are working.
Linking cm3 maybe failed, but I retried having grown the VM from 32MB to 64MB and it worked.

It is something, but I'm not sure what. ?

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 9:18 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Indeed, MS-DOS is just a curiosity at this stage.


Get Outlook for iOS<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7C%7Cc9a9b2b4951d4d942f3808d9e1115d2a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637788286970318190%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZvL3is03zfwZ61RsEcsBGWmgcEZny6OEijrUS6azdk0%3D&reserved=0>
________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 27, 2022 4:18:06 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not much. It is a kinda harsh environment.
Sometimes gdb works, sometimes not.
 (On win10-32 I can break on main, but then gdb fails to resume.)
Sometimes long file names work, sometimes not.
 (I don't try to succeed w/o them.)
Sometimes command lines are restricted to 126 characters, sometimes not.
 (Djgpp is supposed to workaround this, but it does not always work, again, on Win10-32.)
Something like "gcc -c foo.c" exceeds 126 characters because of the underlying cc1/ld.

I will try to try with Win9x and/or FreeDOS or such.
 If it works on more than one maybe declare success.
I don't think any non-Windows MS-DOS from Microsoft has long file names
and I am reluctant to fight that..we'd probably need to start transparently
read/writing .zips or something in m3core.

One unusual thing here is that I am using use threads,
and a customization thereof. Swapping context in timer interrupt.
 https://github.com/modula3/cm3/pull/896/files
but as long as it working in some environments I am satisfied.

I will try to ensure GC is running in DosBox (printf of write(1) about it),
and/or limit memory to induce pressure.

There are interesting things we can do, with more work,
since we are a compiler. For example the generated code
could maintain an easy to walk stack, and print it for assertion failures.

We could even implement a sort of debugger, where we make a call
after every line, to what normally just returns, but can pause
for input, and such.

But I'm not sure how much (more) I am going to put into an MS-DOS port. :)

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:22 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


Aha, interesting. Any more clues?



From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not exactly: MS-DOS, in 32bit Windows.



 - Jay



________________________________

From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.



That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220127/620ca588/attachment-0001.html>

From jayk123 at hotmail.com  Sun Jan 30 08:59:22 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sun, 30 Jan 2022 07:59:22 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
Message-ID: <MWHPR1401MB1951C620C0DED85D89A79B4AE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>

1. Is there an assemption that each call to RTOS.GetMemory is contiguous with prior calls?
2. On the few systems such as this that do not have mmap, might as well we malloc?
  I mean, because, we use sbrk, which is undefined if intermixed with malloc, is deprecated,
  and might as well just use malloc, which is far more standard?
  I will try it out. Or rather calloc for the zeroing.

Here:

PROCEDURE CopySome (): BOOLEAN =
  VAR
    originalPage  := impureCopy.page;
    originalLimit := impureCopy.limit;
    cleanTo       := originalPage + ADRSIZE(PageHdr);
  BEGIN
    LOOP
      IF cleanTo < impureCopy.next THEN
        VAR ptr := impureCopy.next;
        BEGIN
          CleanBetween(cleanTo, ptr, originalPage.desc.clean);
          cleanTo := ptr;
        END;

What is the relationship between cleanTo and ptr?
Granted, this isn't where the assert not forwarded occurs.
It occurs on small ranges, calls to GrayBetween.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 9:18 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Indeed, MS-DOS is just a curiosity at this stage.


Get Outlook for iOS<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7C%7Cc9a9b2b4951d4d942f3808d9e1115d2a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637788286970318190%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZvL3is03zfwZ61RsEcsBGWmgcEZny6OEijrUS6azdk0%3D&reserved=0>
________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 27, 2022 4:18:06 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not much. It is a kinda harsh environment.
Sometimes gdb works, sometimes not.
 (On win10-32 I can break on main, but then gdb fails to resume.)
Sometimes long file names work, sometimes not.
 (I don't try to succeed w/o them.)
Sometimes command lines are restricted to 126 characters, sometimes not.
 (Djgpp is supposed to workaround this, but it does not always work, again, on Win10-32.)
Something like "gcc -c foo.c" exceeds 126 characters because of the underlying cc1/ld.

I will try to try with Win9x and/or FreeDOS or such.
 If it works on more than one maybe declare success.
I don't think any non-Windows MS-DOS from Microsoft has long file names
and I am reluctant to fight that..we'd probably need to start transparently
read/writing .zips or something in m3core.

One unusual thing here is that I am using use threads,
and a customization thereof. Swapping context in timer interrupt.
 https://github.com/modula3/cm3/pull/896/files
but as long as it working in some environments I am satisfied.

I will try to ensure GC is running in DosBox (printf of write(1) about it),
and/or limit memory to induce pressure.

There are interesting things we can do, with more work,
since we are a compiler. For example the generated code
could maintain an easy to walk stack, and print it for assertion failures.

We could even implement a sort of debugger, where we make a call
after every line, to what normally just returns, but can pause
for input, and such.

But I'm not sure how much (more) I am going to put into an MS-DOS port. :)

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:22 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


Aha, interesting. Any more clues?



From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not exactly: MS-DOS, in 32bit Windows.



 - Jay



________________________________

From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.



That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220130/828e23a7/attachment-0001.html>

From jayk123 at hotmail.com  Sun Jan 30 10:38:01 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sun, 30 Jan 2022 09:38:01 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <MWHPR1401MB1951C620C0DED85D89A79B4AE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951C620C0DED85D89A79B4AE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951E09109CA9E805D004A7EE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>

I have a working MS-DOS system and a non-working one.
Non-working is under 32bit Windows 8.1, can allocate only 16MB, just running malloc(64K or such) in a loop.
Working under Win98 MS-DOS w/o booting Windows, can allocate around 300MB.


If I limit RTOS__GetMemory to 16MB on e.g. AMD64_NT I get the "same" problem.
Compiling m3core. With M3C backend, which is a bit inefficient memory/garbage-size,
lots of text concat.


 ***
 *** runtime error:
 ***    <*ASSERT*> failed: NOT h.forwarded
 ***    file "../src/runtime/common/RTCollector.m3", line 607
 ***

 ***
 *** runtime error:
 ***    NEW() was unable to allocate more memory.
 ***    file "../src/runtime/common/RuntimeError.m3", line 63
 ***

but not always in that order.

Is this "just" the behavior under low memory??

Low tech memory constraint:

Posix not actually run:

--- a/m3-libs/m3core/src/runtime/POSIX/RTOSc.c
+++ b/m3-libs/m3core/src/runtime/POSIX/RTOSc.c
@@ -22,13 +22,20 @@ BOOLEAN __cdecl RTOS__Cygwin(void)
 #define MAP_ANON MAP_ANONYMOUS
 #endif

+static INTEGER RTOS__GetMemoryTotal;
+
 ADDRESS
 __cdecl
 RTOS__GetMemory(INTEGER isize)
 {
     WORD_T const size = (WORD_T)isize; // Modula-3 lacks unsigned types, pass as signed and cast.
+    void* p;
+    if (RTOS__GetMemoryTotal > (16 << 20)) return 0; // low memory simulation
     // TODO autoconf/make HAVE_MMAP
-#if defined(ULTRIX)                           || \
+    Scheduler__DisableSwitching ();
+#if defined(__DJGPP__)
+    p = calloc(1, size);
+#elif defined(ULTRIX)                         || \
     defined(ultrix)                           || \
     defined(__ultrix)                         || \
     defined(__ultrix__)                       || \
@@ -50,13 +57,13 @@ RTOS__GetMemory(INTEGER isize)
     // Anecdotal evidence:
     // https://github.com/modula3/cm3/commit/518f93e67ed8f3291a5cd5cf0a39d1fefbc79969
     // https://github.com/modula3/cm3/commit/384b3cc05fcedf4307f077f49cde3d6675b039c6
-    return (ADDRESS)sbrk(size);
-
+    p = sbrk(size);
 #else
-
-    return (ADDRESS)mmap(0, size, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0);
-
+    p = mmap(0, size, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0);
 #endif
+    Scheduler__EnableSwitching ();
+    if (p) RTOS__GetMemoryTotal += size; // low memory simulation
+    return (ADDRESS)p;
 }

 #endif

Actual Win32:

diff --git a/m3-libs/m3core/src/runtime/WIN32/RTOSc.c b/m3-libs/m3core/src/runtime/WIN32/RTOSc.c
index bdece20d2..4dac4792e 100644
--- a/m3-libs/m3core/src/runtime/WIN32/RTOSc.c
+++ b/m3-libs/m3core/src/runtime/WIN32/RTOSc.c
@@ -20,6 +20,8 @@ BOOLEAN __cdecl RTOS__Cygwin (void)
 #endif
 }

+static INTEGER RTOS__GetMemoryTotal;
+
 #if defined (_WIN32) || defined (__CYGWIN__)

 #if 0
@@ -43,7 +45,11 @@ ADDRESS __cdecl RTOS__GetMemory(INTEGER isize)
 ADDRESS __cdecl RTOS__GetMemory(INTEGER isize)
 {
     WORD_T const Size = (WORD_T)isize; // Modula-3 lacks unsigned types, pass as signed and cast.
-    return (ADDRESS)VirtualAlloc(NULL, Size, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
+    void* p;
+    if (RTOS__GetMemoryTotal > (16 << 20)) return 0; // low memory simulation
+    p = VirtualAlloc(NULL, Size, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
+   if (p) RTOS__GetMemoryTotal += Size; // low memory simulation
+   return (ADDRESS)p;
 }

 #elif 0
diff --git a/m3-libs/m3core/src/runtime/common/RTCollector.m3 b/m3-libs/m3core/src/runtime/common/RTCollector.m3
index 16f999ca0..c69b2faed 100644
--- a/m3-libs/m3core/src/runtime/common/RTCollector.m3
+++ b/m3-libs/m3core/src/runtime/common/RTCollector.m3
@@ -596,7 +596,16 @@ PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =
   BEGIN
     WHILE h < he DO
       <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>
-      <* ASSERT NOT h.forwarded *>
+      IF h.forwarded THEN
+        RTIO.PutInt (ORD (r));
+        RTIO.PutText (" ");
+        RTIO.PutAddr (h);
+        RTIO.PutText (" ");
+        RTIO.PutAddr (he);
+        RTIO.PutText ("\n");
+        RTIO.Flush ();
+        <* ASSERT NOT h.forwarded *>
+      END;
       IF h^ # FillHeader1 AND h^ # FillHeaderN THEN
         IF r # PromoteReason.OldImpure OR h.dirty THEN
           h.dirty := FALSE;

PromoteReason is always 4 LargeImpure

I also was printing whenever .forwarded was set, and there is a print to explain the failing assert.

Maybe there is a bug?????

Experiment with other platforms?

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Sunday, January 30, 2022 7:59 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

1. Is there an assemption that each call to RTOS.GetMemory is contiguous with prior calls?
2. On the few systems such as this that do not have mmap, might as well we malloc?
  I mean, because, we use sbrk, which is undefined if intermixed with malloc, is deprecated,
  and might as well just use malloc, which is far more standard?
  I will try it out. Or rather calloc for the zeroing.

Here:

PROCEDURE CopySome (): BOOLEAN =
  VAR
    originalPage  := impureCopy.page;
    originalLimit := impureCopy.limit;
    cleanTo       := originalPage + ADRSIZE(PageHdr);
  BEGIN
    LOOP
      IF cleanTo < impureCopy.next THEN
        VAR ptr := impureCopy.next;
        BEGIN
          CleanBetween(cleanTo, ptr, originalPage.desc.clean);
          cleanTo := ptr;
        END;

What is the relationship between cleanTo and ptr?
Granted, this isn't where the assert not forwarded occurs.
It occurs on small ranges, calls to GrayBetween.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 9:18 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Indeed, MS-DOS is just a curiosity at this stage.


Get Outlook for iOS<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7C%7Cddd0f6f6012d4be09e5508d9e3c6bac2%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637791264951906391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eWtBAhCk92M3n1Is1rzMWWv4IgNmQcOAOnrUY%2F2CfSQ%3D&reserved=0>
________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 27, 2022 4:18:06 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not much. It is a kinda harsh environment.
Sometimes gdb works, sometimes not.
 (On win10-32 I can break on main, but then gdb fails to resume.)
Sometimes long file names work, sometimes not.
 (I don't try to succeed w/o them.)
Sometimes command lines are restricted to 126 characters, sometimes not.
 (Djgpp is supposed to workaround this, but it does not always work, again, on Win10-32.)
Something like "gcc -c foo.c" exceeds 126 characters because of the underlying cc1/ld.

I will try to try with Win9x and/or FreeDOS or such.
 If it works on more than one maybe declare success.
I don't think any non-Windows MS-DOS from Microsoft has long file names
and I am reluctant to fight that..we'd probably need to start transparently
read/writing .zips or something in m3core.

One unusual thing here is that I am using use threads,
and a customization thereof. Swapping context in timer interrupt.
 https://github.com/modula3/cm3/pull/896/files
but as long as it working in some environments I am satisfied.

I will try to ensure GC is running in DosBox (printf of write(1) about it),
and/or limit memory to induce pressure.

There are interesting things we can do, with more work,
since we are a compiler. For example the generated code
could maintain an easy to walk stack, and print it for assertion failures.

We could even implement a sort of debugger, where we make a call
after every line, to what normally just returns, but can pause
for input, and such.

But I'm not sure how much (more) I am going to put into an MS-DOS port. :)

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:22 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


Aha, interesting. Any more clues?



From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not exactly: MS-DOS, in 32bit Windows.



 - Jay



________________________________

From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.



That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220130/7d18c898/attachment-0001.html>

From jayk123 at hotmail.com  Sun Jan 30 10:54:06 2022
From: jayk123 at hotmail.com (Jay K)
Date: Sun, 30 Jan 2022 09:54:06 +0000
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.
In-Reply-To: <MWHPR1401MB1951E09109CA9E805D004A7EE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>
References: <MWHPR1401MB195126995FD2EC61F38C89C0E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB275554457D17651583993847B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951A0FDEB409C2085A94B3BE6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27557C94110A441B712A4F2CB2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951ACD919B86738B27F18C8E6209@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <SYBP282MB27556798721D9E1FA4E4CEA7B2209@SYBP282MB2755.AUSP282.PROD.OUTLOOK.COM>
 <MWHPR1401MB1951C620C0DED85D89A79B4AE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>
 <MWHPR1401MB1951E09109CA9E805D004A7EE6249@MWHPR1401MB1951.namprd14.prod.outlook.com>
Message-ID: <MWHPR1401MB1951B2A208D52DB79095D3A7E6249@MWHPR1401MB1951.namprd14.prod.outlook.com>

Yes this appears to be nested within out of memory.
But I am not sure that excuses it.
Here it is (at the end) with the limit raised to 32MB.

Maybe the real trouble is here:
 cm3!RTHooks__AllocateTracedRef+0x51 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 118]
 cm3!ETimer__New+0x79 [C:\s\cm3\m3-libs\libm3\src\etimer\ETimer.m3 @ 45]
 cm3!WebFile__Dump+0x107 [C:\s\cm3\m3-sys\cm3\src\WebFile.m3 @ 53]

Where a finalizer requires allocating memory.

PROCEDURE Dump () =
  VAR x: InfoEntry;
  BEGIN
    IF (new_info = NIL) THEN RETURN END;
    x := new_info;
    new_info := NIL;  (* in case we die while updating *)

    ETimer.Push (ETimer.New ("updating web info"));
      DumpFile (x, ParseFile (Inhale (), x));
    ETimer.Pop ();
  END Dump;

not just for the timer, but the DumpFile line looks dubious.

Well maybe you know, here:

 cm3!Main__CleanUp+0x1b6 [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 209]
 cm3!RTProcess__InvokeExitors+0x81 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 35]

    Process.RegisterExitor (CleanUp);

there should be some differentiation between why existing is occuring.
Like some systems have "quick exit".

0:000> k
Child-SP          RetAddr               Call Site
000000a3`c9efc498 00007ff6`a22a5458     KERNELBASE!DebugBreak+0x2
 cm3!RTOS__Crash+0x68 [C:\s\cm3\m3-libs\m3core\src\runtime\WIN32\RTOS.m3 @ 33]
 cm3!RTProcess__Crash+0x70 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 66]
 cm3!RTError__EndError+0x41 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119]
 cm3!RTError__MsgS+0xa4 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41]
 cm3!RTException__Crash+0x36c [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81]
 cm3!RTException__DefaultBackstop+0x101 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39]
 cm3!RTException__InvokeBackstop+0x3d [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25]
 cm3!RTException__Raise+0x9e [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90]
 cm3!RTException__DefaultBackstop+0x1ce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49]
 cm3!RTException__InvokeBackstop+0x3d [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25]
 cm3!RTException__Raise+0x9e [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90]
 cm3!RTHooks__AssertFailed+0xbf [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 129]
 cm3!RTCollector__GrayBetween+0x14e [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 609]
 cm3!RTCollector__PromotePage+0x5f1 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 581]
 cm3!RTCollector__Move+0x48a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 460]
 cm3!RTHeapMap__Walk+0x91f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHeapMap.m3 @ 203]
 cm3!RTHeapMap__DoWalkRef+0xc8 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHeapMap.m3 @ 63]
 cm3!RTHeapMap__WalkRef+0x1a7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHeapMap.m3 @ 49]
 cm3!RTCollector__CleanBetween+0x255 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 1113]
 cm3!RTCollector__CopySome+0xe7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 1060]
 cm3!RTCollector__CollectSomeInStateOne+0x1c [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 927]
 cm3!RTCollector__CollectSome+0xb7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 741]
 cm3!RTHeapRep__Crash+0x4e [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTCollector.m3 @ 1621]
 cm3!RTProcess__Crash+0x63 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 64]
 cm3!RTError__EndError+0x41 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119]
 cm3!RTError__MsgS+0xa4 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41]
 cm3!RTException__Crash+0x36c [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81]
 cm3!RTException__DefaultBackstop+0x101 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39]
 cm3!RTException__InvokeBackstop+0x3d [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25]
 cm3!RTException__Raise+0x9e [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90]
 cm3!RTException__DefaultBackstop+0x1ce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49]
 cm3!RTException__InvokeBackstop+0x3d [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25]
 cm3!RTException__Raise+0x9e [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90]
 cm3!RTHooks__Raise+0xb4 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 80]
 cm3!RuntimeError__Raise+0x6a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RuntimeError.m3 @ 63]
 cm3!RTHooks__AllocateTracedRef+0x51 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 118]
 cm3!ETimer__New+0x79 [C:\s\cm3\m3-libs\libm3\src\etimer\ETimer.m3 @ 45]
 cm3!WebFile__Dump+0x107 [C:\s\cm3\m3-sys\cm3\src\WebFile.m3 @ 53]
 cm3!Main__CleanUp+0x1b6 [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 209]
 cm3!RTProcess__InvokeExitors+0x81 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 35]
 cm3!RTProcess__Crash+0x5e [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 64]
 cm3!RTError__EndError+0x41 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119]
 cm3!RTError__MsgS+0xa4 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41]
 cm3!RTException__Crash+0x36c [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81]
 cm3!RTException__DefaultBackstop+0x101 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39]
 cm3!RTException__InvokeBackstop+0x3d [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25]
 cm3!RTException__Raise+0x9e [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90]
 cm3!RTException__DefaultBackstop+0x1ce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49]
 cm3!RTException__InvokeBackstop+0x3d [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25]
 cm3!RTException__Raise+0x9e [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90]
 cm3!RTHooks__Raise+0xb4 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 80]
 cm3!RuntimeError__Raise+0x6a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RuntimeError.m3 @ 63]
 cm3!RTHooks__AllocateTracedObj+0x51 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 125]

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Sunday, January 30, 2022 9:38 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

I have a working MS-DOS system and a non-working one.
Non-working is under 32bit Windows 8.1, can allocate only 16MB, just running malloc(64K or such) in a loop.
Working under Win98 MS-DOS w/o booting Windows, can allocate around 300MB.


If I limit RTOS__GetMemory to 16MB on e.g. AMD64_NT I get the "same" problem.
Compiling m3core. With M3C backend, which is a bit inefficient memory/garbage-size,
lots of text concat.


 ***
 *** runtime error:
 ***    <*ASSERT*> failed: NOT h.forwarded
 ***    file "../src/runtime/common/RTCollector.m3", line 607
 ***

 ***
 *** runtime error:
 ***    NEW() was unable to allocate more memory.
 ***    file "../src/runtime/common/RuntimeError.m3", line 63
 ***

but not always in that order.

Is this "just" the behavior under low memory??

Low tech memory constraint:

Posix not actually run:

--- a/m3-libs/m3core/src/runtime/POSIX/RTOSc.c
+++ b/m3-libs/m3core/src/runtime/POSIX/RTOSc.c
@@ -22,13 +22,20 @@ BOOLEAN __cdecl RTOS__Cygwin(void)
 #define MAP_ANON MAP_ANONYMOUS
 #endif

+static INTEGER RTOS__GetMemoryTotal;
+
 ADDRESS
 __cdecl
 RTOS__GetMemory(INTEGER isize)
 {
     WORD_T const size = (WORD_T)isize; // Modula-3 lacks unsigned types, pass as signed and cast.
+    void* p;
+    if (RTOS__GetMemoryTotal > (16 << 20)) return 0; // low memory simulation
     // TODO autoconf/make HAVE_MMAP
-#if defined(ULTRIX)                           || \
+    Scheduler__DisableSwitching ();
+#if defined(__DJGPP__)
+    p = calloc(1, size);
+#elif defined(ULTRIX)                         || \
     defined(ultrix)                           || \
     defined(__ultrix)                         || \
     defined(__ultrix__)                       || \
@@ -50,13 +57,13 @@ RTOS__GetMemory(INTEGER isize)
     // Anecdotal evidence:
     // https://github.com/modula3/cm3/commit/518f93e67ed8f3291a5cd5cf0a39d1fefbc79969
     // https://github.com/modula3/cm3/commit/384b3cc05fcedf4307f077f49cde3d6675b039c6
-    return (ADDRESS)sbrk(size);
-
+    p = sbrk(size);
 #else
-
-    return (ADDRESS)mmap(0, size, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0);
-
+    p = mmap(0, size, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0);
 #endif
+    Scheduler__EnableSwitching ();
+    if (p) RTOS__GetMemoryTotal += size; // low memory simulation
+    return (ADDRESS)p;
 }

 #endif

Actual Win32:

diff --git a/m3-libs/m3core/src/runtime/WIN32/RTOSc.c b/m3-libs/m3core/src/runtime/WIN32/RTOSc.c
index bdece20d2..4dac4792e 100644
--- a/m3-libs/m3core/src/runtime/WIN32/RTOSc.c
+++ b/m3-libs/m3core/src/runtime/WIN32/RTOSc.c
@@ -20,6 +20,8 @@ BOOLEAN __cdecl RTOS__Cygwin (void)
 #endif
 }

+static INTEGER RTOS__GetMemoryTotal;
+
 #if defined (_WIN32) || defined (__CYGWIN__)

 #if 0
@@ -43,7 +45,11 @@ ADDRESS __cdecl RTOS__GetMemory(INTEGER isize)
 ADDRESS __cdecl RTOS__GetMemory(INTEGER isize)
 {
     WORD_T const Size = (WORD_T)isize; // Modula-3 lacks unsigned types, pass as signed and cast.
-    return (ADDRESS)VirtualAlloc(NULL, Size, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
+    void* p;
+    if (RTOS__GetMemoryTotal > (16 << 20)) return 0; // low memory simulation
+    p = VirtualAlloc(NULL, Size, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
+   if (p) RTOS__GetMemoryTotal += Size; // low memory simulation
+   return (ADDRESS)p;
 }

 #elif 0
diff --git a/m3-libs/m3core/src/runtime/common/RTCollector.m3 b/m3-libs/m3core/src/runtime/common/RTCollector.m3
index 16f999ca0..c69b2faed 100644
--- a/m3-libs/m3core/src/runtime/common/RTCollector.m3
+++ b/m3-libs/m3core/src/runtime/common/RTCollector.m3
@@ -596,7 +596,16 @@ PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =
   BEGIN
     WHILE h < he DO
       <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>
-      <* ASSERT NOT h.forwarded *>
+      IF h.forwarded THEN
+        RTIO.PutInt (ORD (r));
+        RTIO.PutText (" ");
+        RTIO.PutAddr (h);
+        RTIO.PutText (" ");
+        RTIO.PutAddr (he);
+        RTIO.PutText ("\n");
+        RTIO.Flush ();
+        <* ASSERT NOT h.forwarded *>
+      END;
       IF h^ # FillHeader1 AND h^ # FillHeaderN THEN
         IF r # PromoteReason.OldImpure OR h.dirty THEN
           h.dirty := FALSE;

PromoteReason is always 4 LargeImpure

I also was printing whenever .forwarded was set, and there is a print to explain the failing assert.

Maybe there is a bug?????

Experiment with other platforms?

 - Jay

________________________________
From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Sent: Sunday, January 30, 2022 7:59 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

1. Is there an assemption that each call to RTOS.GetMemory is contiguous with prior calls?
2. On the few systems such as this that do not have mmap, might as well we malloc?
  I mean, because, we use sbrk, which is undefined if intermixed with malloc, is deprecated,
  and might as well just use malloc, which is far more standard?
  I will try it out. Or rather calloc for the zeroing.

Here:

PROCEDURE CopySome (): BOOLEAN =
  VAR
    originalPage  := impureCopy.page;
    originalLimit := impureCopy.limit;
    cleanTo       := originalPage + ADRSIZE(PageHdr);
  BEGIN
    LOOP
      IF cleanTo < impureCopy.next THEN
        VAR ptr := impureCopy.next;
        BEGIN
          CleanBetween(cleanTo, ptr, originalPage.desc.clean);
          cleanTo := ptr;
        END;

What is the relationship between cleanTo and ptr?
Granted, this isn't where the assert not forwarded occurs.
It occurs on small ranges, calls to GrayBetween.

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 9:18 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Indeed, MS-DOS is just a curiosity at this stage.


Get Outlook for iOS<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7C%7C8fa054598e8e4ced236908d9e3d484ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637791324193727225%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=J0xAEQQ9b%2FmpoD6v9Ve5n26%2FnwcMOMs%2B7zrlJUegOvg%3D&reserved=0>
________________________________
From: Jay K <jayk123 at hotmail.com>
Sent: Thursday, January 27, 2022 4:18:06 AM
To: Tony Hosking <antony.hosking at anu.edu.au>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not much. It is a kinda harsh environment.
Sometimes gdb works, sometimes not.
 (On win10-32 I can break on main, but then gdb fails to resume.)
Sometimes long file names work, sometimes not.
 (I don't try to succeed w/o them.)
Sometimes command lines are restricted to 126 characters, sometimes not.
 (Djgpp is supposed to workaround this, but it does not always work, again, on Win10-32.)
Something like "gcc -c foo.c" exceeds 126 characters because of the underlying cc1/ld.

I will try to try with Win9x and/or FreeDOS or such.
 If it works on more than one maybe declare success.
I don't think any non-Windows MS-DOS from Microsoft has long file names
and I am reluctant to fight that..we'd probably need to start transparently
read/writing .zips or something in m3core.

One unusual thing here is that I am using use threads,
and a customization thereof. Swapping context in timer interrupt.
 https://github.com/modula3/cm3/pull/896/files
but as long as it working in some environments I am satisfied.

I will try to ensure GC is running in DosBox (printf of write(1) about it),
and/or limit memory to induce pressure.

There are interesting things we can do, with more work,
since we are a compiler. For example the generated code
could maintain an easy to walk stack, and print it for assertion failures.

We could even implement a sort of debugger, where we make a call
after every line, to what normally just returns, but can pause
for input, and such.

But I'm not sure how much (more) I am going to put into an MS-DOS port. :)

 - Jay

________________________________
From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:22 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.


Aha, interesting. Any more clues?



From: Jay K <jayk123 at hotmail.com>
Date: Thursday, January 27, 2022 at 12:21 AM
To: Tony Hosking <antony.hosking at anu.edu.au>, m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.

Not exactly: MS-DOS, in 32bit Windows.



 - Jay



________________________________

From: Tony Hosking <antony.hosking at anu.edu.au>
Sent: Wednesday, January 26, 2022 1:14 PM
To: Jay K <jayk123 at hotmail.com>; m3devel <m3devel at elegosoft.com>
Subject: Re: RTCollector.m3 assert NOT h.forwarded under MS-DOS.



That?s a pretty severe GC error. Ae you saying that the collector is failing on 32-bit Windows 10?



From: M3devel <m3devel-bounces at elegosoft.com> on behalf of Jay K <jayk123 at hotmail.com>
Date: Wednesday, January 26, 2022 at 11:19 PM
To: m3devel <m3devel at elegosoft.com>
Subject: [M3devel] RTCollector.m3 assert NOT h.forwarded under MS-DOS.

cm3 is running kinda on MS-DOS. Decades late, sorry. :)

I think I have user threads working even.



In DosBos-X it can generate all the cpp files for m3core.

And then fails, because, autoconf not found.

I can either fix the autotools path, or run compiler more directly.





Under 32bit Windows 10 it doesn't get quite as far:

***    <*ASSERT*> failed: NOT h.forwarded

***    file "../src/runtime/common/RTCollector.m3", line 599

***



PROCEDURE GrayBetween (h, he: RefHeader; r: PromoteReason) =

  BEGIN

    WHILE h < he DO

      <* ASSERT Word.And (LOOPHOLE (h, INTEGER), 3) = 0 *>

      <* ASSERT NOT h.forwarded *>

      IF h^ # FillHeader1 AND h^ # FillHeaderN THEN

        IF r # PromoteReason.OldImpure OR h.dirty THEN

          h.dirty := FALSE;

          h.gray := TRUE;

        END;

      END;

      INC(h, ADRSIZE(Header) + ReferentSize(h));

    END;

  END GrayBetween;



I think I saw similar, on IA64_HPUX, but, that was when I had

the system as little endian, whereas IA64_HPUX is big endian.



That is not likely the problem here, especially since it works under DosBox-X.



Any ideas?



Well, it isn't sbrk vs. mmap at least. Djgpp lacks mmap and sbrk is being used.



It is possible the DosBox-X environment has more memory and the garbage collector

does not work there either.



Thank you,

 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20220130/f41ee218/attachment-0001.html>