[M3devel] the formsedit crash

Mon Jul 27 19:17:51 CEST 2009

 > I don't know of a way to increase the mainline thread stack while it is running.  

On NT you cannot. The stack size is set when the thread is created. Once the thread is created, that is it.
Typical default thread sizes on 32bit NT are 256K and 1MB. Maybe twice that for 64bit.
I think there might be a minimum that small requests get rounded up to.
VirtualAlloc basically allocates in 64K chunks so you can't likely get smaller than 64K, or 60K if you consider the guard page. Some experimentation would be informative.

I've seen gcc stack overflow only on non-Linux..because the default stack on Linux is huge, like 8MB.
I guess that's what you get in historically single threaded environments..

If you write an NT kernel driver, then you generally have like 3 4K pages maximum, a very different environment.

Basically I consider this area fairly unpredictable.
Did I create the thread or did someone else create it and call me?
How much stack is required by the code I'm going to call.

In the absence of any solutions, best advise for the programmer is to generally conserve stack.
If you must recurse, recurse only to a maximum amount determined by your code, not determined by user input.
Otherwise large user input will cause you to fail in a hard to handle way.

Fixed sized character buffers of 256, 1024, MAX_PATH, etc. are not a good thing to use.
  Doubly (or even quadruply) bad with Unicode.
Despite being very fast. Another option is to use a small buffer like 64 and then heap alloc if the data is large. Or just heap alloc and have a smooth if slow performance curve against data size.

At least with the heap, it is pretty much limited by address space, and running out of heap at least in C is very well defined -- you get NULL from malloc. Many people know to check for that.

 - Jay

----------------------------------------
> From: jay.krell at cornell.edu
> To: rcoleburn at scires.com; m3devel at elegosoft.com
> Subject: RE: [M3devel] the formsedit crash
> Date: Mon, 27 Jul 2009 16:33:43 +0000
>
>
>> a command line parameter to set the stack size!
>
> This is proof that the mechanism is very flawed.
> Imagine if all programs accepted a command line parameter to set their stack size, and chosing the right number was critical to their successful operation.
>
> !!!
>
> - Jay
>
>
>
> ________________________________
>> Date: Mon, 27 Jul 2009 12:29:41 -0400
>> From: rcoleburn at scires.com
>> To: m3devel at elegosoft.com
>> Subject: Re: [M3devel] the formsedit crash
>>
>>
>>
>>
>>
>>
>>
>> In my applications, I added code to accept a command line parameter for increasing the stack size. Internally, I call the procedures from the Thread interface to increase the stack size. I had to do this to avoid running out of stack, esp. when using Trestle/FormsVBT. I found that I had to multiply the stack size by a factor of 7 for most of my stuff to work. Note that this increase applies to new threads, not to the mainline thread. I don't know of a way to increase the mainline thread stack while it is running.
>>
>>
>>
>> If there is a change to the default stack size, anyone who does similar tricks will need to modify their code, or their shortcuts, or scripts to avoid making the stack too big.
>>
>>
>>
>> See the code snippet below:
>>
>>
>>
>> (* stack *)
>> IF pp.keywordPresent("-stack")
>> THEN (* *)
>> TRY (* *)
>> stackMult := pp.getNextInt(min := 1, max := 100);
>> EXCEPT
>> | ParseParams.Error => (* *)
>> pp.error("A number in the range [1..100] must be specified as\n"
>> & "the stack size multiplier after the -stack option.\n");
>> END; (* try *)
>> END; (* if *)
>> IF stackMult> 1
>> THEN
>> WITH default = Thread.GetDefaultStackSize(),
>> desired = default * stackMult,
>> increase = desired - default
>> DO
>> PutText("Increasing default stack size by a factor of " &
>> Fmt.Int(stackMult) & " (" & Fmt.Int(default) & ">>> " &
>> Fmt.Int(desired) & ").\n");
>> Thread.IncDefaultStackSize(increase);
>> END; (* with *)
>> WITH default = Thread.GetDefaultStackSize()
>> DO
>> PutText("Thread stacks are now " & Fmt.Int(default) & " bytes.\n");
>> END; (* with *)
>> END; (* if *)
>>
>>
>>
>> Regards,
>>
>> Randy Coleburn
>>
>>>>> Jay K 7/27/2009 11:24 AM>>>
>>
>> I don't think this is a stack overflow but I will try that.
>> We should probably drastically increase the defaults.
>> Like make them match whatever the underlying platform uses, or just make
>> sure that unless the user intercedes, that we don't either.
>>
>>
>> The whole notion of a limited size stack is pretty flawed imho.
>> And much worse if you engage in reducing it from the default.
>> How much stack do I need to leave for fopen to work?
>> What do I do when I run out of stack? On NT you can detect and handle it, but it is too tricky.
>> Personally I try to use very little stack, and don't use recursion - if I must use a "heap allocated stack" (e.g. std::stack<> in C++).
>> I understand though that the machine stack is tremendously faster than anything heap-based (at least in non-gc environments, gc heap alloc can be fast, if you don't mind the extra work for the gc to later clean it up..and if the usage is actually lifo...)
>>
>>
>> Granted, if I'm "just doing a bunch of math" and don't call into code I don't control, then it becomes more tenable. Still hard to pick a size that is portable, though multiplying by sizeof(void*) helps.
>> What if I use TRY? That uses a highly platform-specific and often fairly large amount of stack.
>>
>>
>> - Jay
>>
>>
>> ----------------------------------------
>>> Date: Mon, 27 Jul 2009 16:53:08 +0200
>>> From: wagner at elegosoft.com
>>> To: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] the formsedit crash
>>>
>>> Quoting Tony Hosking :
>>>
>>>> Not sure if doing it that way works. Current default stack size set in
>>>> ThreadPThread.m3 is 4096 * ADRSIZE(Word.T). You'll need to try
>>>> changing it there.
>>>
>>> There used to be a procedure IncreaseDefaultStackSize. IITR we needed
>>> that for several M3 applications on some platforms several years ago.
>>>
>>> But probably we should also set the default for a target platform
>>> so that all our own applications are able to run.
>>>
>>> Olaf
>>> --
>>> Olaf Wagner -- elego Software Solutions GmbH
>>> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
>>> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
>>> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
>>> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
>>>