Why does a stack overflow cause a VCL application to terminate?

[Der schöne Günther 316]

Consider this complete VCL application (Form1 & Button1):

unit Unit1;

interface uses
	System.SysUtils, System.Classes,
	Vcl.Forms, Vcl.StdCtrls, Vcl.Controls;

type
	TForm1 = class(TForm)
		Button1: TButton;
		procedure Button1Click(Sender: TObject);
	end;

var
	Form1: TForm1;

implementation

{$R *.dfm}

procedure causeStackOverflow();
begin
	causeStackOverflow();
end;

procedure TForm1.Button1Click(Sender: TObject);
begin
	causeStackOverflow();
end;

end.

Clicking the button causes the application to hang for a bit, then the debugger will break on a EStackOverflow exception.

That's totally expected.

 

But directly after that, the application tries to display its regular error dialog, causing an access violation and then silently crashing.

This is the callstack:

Quote

Vcl.Forms.TApplication.MessageBox('Stack overflow.','Project1',16)
Vcl.Forms.TApplication.ShowException$ActRec.$0$Body
System.Classes.TThread.Synchronize($A29C0,False,False)
System.Classes.TThread.Synchronize(???,TApplication.ShowException$ActRec($2CE3234) as TThreadProcedure)
Vcl.Forms.TApplication.ShowException(???)
Vcl.Forms.TApplication.HandleException(???)
Vcl.Controls.TWinControl.MainWndProc(???)

It will then cause an access violation in user32.dll repeatedly and then crash.

 

I have no idea why. 64 Bit is fine, by the way. It just happens with 32 Bit .exe.

[Attila Kovacs 629]

Perhaps it's because x64 parameters are stored in registers instead of on the stack. Could you check the assembly code and add more parameters to see if x64 fails as well?

[David Heffernan 2345]

The 32 bit behaviour is preferable. Why do you want to do anything after a stack overflow? 

[Remy Lebeau 1394]

Makes sense why it would fail.  A stack overflow means there is no more stack space available to push new data onto.  In x86, all of those extra function calls are going to keep trying to push data onto the call stack.  Eventually something has to give way.

 

Moral of the story - don't overload the call stack in the first place!

Edited May 5, 2023 by Remy Lebeau

[Der schöne Günther 316]

20 minutes ago, Attila Kovacs said:

add more parameters to see if x64 fails as well?

Good catch, it does.

 

11 minutes ago, Remy Lebeau said:

Makes sense why it would fail.  A stack overflow means there is no more stack space available to push new data onto

Absolutely, but I expected it to recover from it by popping the stack to the next exception handler and proceeding as usual.

 

 

Does that mean a stack overflow in the main thread should generally be seen as non-recoverable and game over?

Edited May 5, 2023 by Der schöne Günther

[Remy Lebeau 1394]

7 hours ago, Der schöne Günther said:

Absolutely, but I expected it to recover from it by popping the stack to the next exception handler and proceeding as usual.

It's possible that some stack unwinding does occur, but the call stack you provided earlier clearly shows that the exception handler makes a bunch of function calls, which could easily eat up whatever stack space was freed up.

Quote

Does that mean a stack overflow in the main thread should generally be seen as non-recoverable and game over?

Pretty much, yes.

 

A thread has a fairly limited call stack available to it.  The stack can grow only so much before it overflows.  That is why it is not a good idea to put a lot of data on the stack in the first place.  However, you can set the limits in the project options, if you need them higher.  Just know that doing so can affect all threads created at runtime (unless overwritten in code on a per-thread basis).

Edited May 6, 2023 by Remy Lebeau

[Dalija Prasnikar 1396]

14 hours ago, Der schöne Günther said:

Absolutely, but I expected it to recover from it by popping the stack to the next exception handler and proceeding as usual.

As the name says, stack overflow overflows the memory buffer reserved for stack. In other words, it writes data in memory that does not belong to the particular stack buffer. This can corrupt parts of memory used for something else and this is also why stack overflow exception is not recoverable and can crash the application.

[David Heffernan 2345]

4 hours ago, Dalija Prasnikar said:

This can corrupt parts of memory used for something else and this is also why stack overflow exception is not recoverable and can crash the application.

Isn't the top of the stack protected by guard pages? 

[Attila Kovacs 629]

19 hours ago, Der schöne Günther said:

Absolutely, but I expected it to recover from it by popping the stack to the next exception handler and proceeding as usual.

 

There is no such thing as 'popping back,' and it's highly likely that the last return address is already missing or sitting at the wrong stack position.

 

1 hour ago, David Heffernan said:

Isn't the top of the stack protected by guard pages? 

It only guards against small increases in the stack, not large ones.

 

[Dalija Prasnikar 1396]

4 hours ago, David Heffernan said:

Isn't the top of the stack protected by guard pages? 

Now we are going into territory I don't know well. 

 

When guard pages are used, recovering from stack overflow exception requires some "manual" handling. While I don't know what Delphi actually does when it happens, it is fairly obvious that stack overflow exception corrupts memory as applications die almost immediately afterwards due to random AV exceptions or just get killed by the OS.  

[Rollo62 536]

Have you tried to encapsule this in try-finally-except, in the highest level caller ?

I doubt that this will catch any possible failure cases, but maybe if it does, this could at least help to close the app gracefully.

Edited May 8, 2023 by Rollo62

[David Heffernan 2345]

5 minutes ago, Rollo62 said:

Have you tried to encapsule this in try-finally-except, in the highest level caller ?

I doubt that this will catch any possible failure cases, but maybe if it does, this could at least help to close the app gracefully.

VCL apps already do this 

[Rollo62 536]

Thanks, so that means it would be able to catch this in the global exception handler ?

https://www.swissdelphicenter.ch/de/showcode.php?id=819

https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Default_Exception_Handling_in_VCL

[David Heffernan 2345]

Not sure what you mean by "this" 

Edited May 8, 2023 by David Heffernan

[Rollo62 536]

this = the AV exception

 

Like I said, I doubt this can be catched safely, because the system is highly unstable probably.

 

The better way would be to determine and limit the stack size usage in the first place.

Maybe this is also interesting reading:

https://blog.grijjy.com/2019/01/25/allocation-free-collections/

https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Default_Exception_Handling_in_VCL

https://stackoverflow.com/questions/6150018/what-is-a-safe-maximum-stack-size-or-how-to-measure-use-of-stack

https://itecnote.com/tecnote/delphi-a-safe-maximum-stack-size-or-how-to-measure-use-of-stack/

https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Memory_allocation_sizes_(Delphi)

https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Reserved_address_space_for_resources_(Delphi)

https://www.thoughtco.com/understanding-memory-allocation-in-delphi-1058464

http://unigui.com/doc/online_help/thread-stack-size.htm

 

Not sure how to determine the current used and available stack size in the modern Windows ( and other platforms ) world, not have needed that for the last 15 years or so.

Edited May 8, 2023 by Rollo62