rease ... at ReturnAddress

[stijnsanders 35]

I read this blog post:

https://blog.dummzeuch.de/2024/06/04/an-improved-abstract-error-handler-for-delphi/
and noticed I had something similar, but noticed that Delphi since some recent version introduced a function ReturnAddress that is generally available, it doesn't even have code in System.pas so I suspect it's something the compiler introduces.

I also notice older Delphi versions don't have this yet so I'm not sure at what version this got introduced. Does anybody know or can check?

(Also I guess using ReturnAddress would future-proof your code in case the [ebp+...] offset changes again, and also should be available on other processortypes?)

[Stefan Glienke 1974]

XE2 - see https://bitbucket.org/sglienke/spring4d/src/2dbce92195d699d51fc99dd226c4698748ec8ef9/Source/Base/Spring.pas#lines-3140

 

Anyhow the ReturnAddress function would not help here because it would point at the location in System._AbstractError which is the function that calls AbstractErrorProc.

This improved handler is a highly brittle hack that might work depending on your compiler settings. It does not force specific compiler settings for the u_dzAbstractHandler.pas unit which also changes where the return address 2 calls up is to be found and it also does not even work for x64.

 

I also don't share Thomas' assessment that this change came in Tokyo but I rather suspect that his stackframe settings were different between his tries on different Delphi versions. The code that is responsible for the abstract error did not change between Delphi XE (the oldest version I can check right now) and 12.

 

The best way would probably be to use Caller(2) from JclDebug which does a proper stack walking to determine the return address.

Edited June 5 by Stefan Glienke

[dummzeuch 1456]

6 hours ago, Stefan Glienke said:

I also don't share Thomas' assessment that this change came in Tokyo but I rather suspect that his stackframe settings were different between his tries on different Delphi versions.

You caught me on that: I didn't verify the stack frame settings, but I have just done so.

The (new) code works for Delphi 2007 and Delphi 12(.1) with or without stack frames. I didn't check the other versions.

 

Can you think of any other compiler settings I'd need to check?

 

6 hours ago, Stefan Glienke said:

and it also does not even work for x64.

My target platform is Win32 only. It doesn't even compile for anything else because of the asm syntax anyway.

 

6 hours ago, Stefan Glienke said:

The best way would probably be to use Caller(2) from JclDebug which does a proper stack walking to de

termine the return address.

... which would require me to add JclDebug and its dependencies to all programs that use this simple unit. For most of the programs at work this wouldn't make any difference because they use JclDebug anyway, but my personal projects don't.

 

Hm, is JclDebug even compatible to X64? Or other target platforms? I have never checked.

Edited June 5 by dummzeuch

[dummzeuch 1456]

25 minutes ago, dummzeuch said:

the (new) code works for Delphi 2007 and Delphi 12(.1) with or without stack frames. I didn't check the other versions.

The difference seems to be the RTL code. In Delphi 2007 it does not create a stack frame, regardess if using debug dcus or not:

@AbstractError:
00402EEC 833D28B0400000   cmp dword ptr [$0040b028],$00
00402EF3 7406             jz $00402efb
00402EF5 FF1528B04000     call dword ptr [$0040b028]

In Delphi 12 it does, again regardless if using debug dcus or not:

System.pas.11789: begin
00AD4A68 55               push ebp
00AD4A69 8BEC             mov ebp,esp
System.pas.11790: if Assigned(AbstractErrorProc) then
00AD4A6B 833D3840AF0000   cmp dword ptr [$00af4038],$00
00AD4A72 7406             jz $00ad4a7a
System.pas.11791: AbstractErrorProc;
00AD4A74 FF153840AF00     call dword ptr [$00af4038]

So, yes the difference seems to be the compiler settings for stack frames, but not in my code but in the System unit.

 

Now the question is when this was changed.

Edited June 5 by dummzeuch

[Remy Lebeau 1330]

3 hours ago, dummzeuch said:

So, yes the difference seems to be the compiler settings for stack frames, but not in my code but in the System unit.

 

Now the question is when this was changed.

XE2.  Prior to that, _AbstractError() used hand-written assembly code that didn't use a stack frame.

 

In Delphi 5 (and probably earlier, I can't check), it looked like this:

procedure       _AbstractError;
asm
        CMP     AbstractErrorProc, 0
        JE      @@NoAbstErrProc
        CALL    AbstractErrorProc

@@NoAbstErrProc:
        MOV     EAX,210
        JMP     _RunError
end;

In Delphi 6, some extra platforms were added:

procedure       _AbstractError;
{$IFDEF PC_MAPPED_EXCEPTIONS}
asm
        MOV     EAX,210
        JMP     _RunError
end;
{$ELSE}
{$IFDEF PIC}
begin
  if Assigned(AbstractErrorProc) then
    AbstractErrorProc;
  _RunError(210);  // loses return address
end;
{$ELSE}
asm
        CMP     AbstractErrorProc, 0
        JE      @@NoAbstErrProc
        CALL    AbstractErrorProc

@@NoAbstErrProc:
        MOV     EAX,210
        JMP     _RunError
end;
{$ENDIF}
{$ENDIF}

Then somewhere between D7-D2006, the PC_MAPPED_EXCEPTIONS branch was dropped:

procedure       _AbstractError;
{$IFDEF PIC}
begin
  if Assigned(AbstractErrorProc) then
    AbstractErrorProc;
  _RunError(210);  // loses return address
end;
{$ELSE}
asm
        CMP     AbstractErrorProc, 0
        JE      @@NoAbstErrProc
        CALL    AbstractErrorProc

@@NoAbstErrProc:
        MOV     EAX,210
        JMP     _RunError
end;
{$ENDIF}

Then in XE, the assembly code was restricted to just 386 CPUs and tweaked to include stack alignment, but still no stack frame:

procedure _AbstractError;
{$IF (not defined(CPU386)) or defined(PIC)}
begin

  if Assigned(AbstractErrorProc) then
    AbstractErrorProc;
  _RunError(210);  // loses return address
end;
{$ELSE}
asm
        CMP     AbstractErrorProc, 0
        JE      @@NoAbstErrProc
{$IFDEF ALIGN_STACK}
        SUB    ESP, 12
{$ENDIF ALIGN_STACK}
        CALL    AbstractErrorProc
{$IFDEF ALIGN_STACK}
        ADD    ESP, 12
{$ENDIF ALIGN_STACK}
@@NoAbstErrProc:
        MOV     EAX,210
        JMP     _RunError
end;
{$IFEND}

And then finally in XE2, the assembly code was eliminated completely:

procedure _AbstractError;
begin
  if Assigned(AbstractErrorProc) then
    AbstractErrorProc;
  RunErrorAt(210, ReturnAddress);
end;

Which is why it is now subject to the compiler's setting for stack frames when compiling System.pas.

Edited June 5 by Remy Lebeau

[corneliusdavid 183]

6 hours ago, stijnsanders said:

https://blog.dummzeuch.de/2024/06/04/an-improved-abstract-error-handler-for-delphi/

The solution for this is genius! Thanks, @dummzeuch for that blog article and the library!

[dummzeuch 1456]

1 hour ago, Remy Lebeau said:

XE2.  Prior to that, _AbstractError() used hand-written assembly code that didn't use a stack frame.

Good to know. I just went through several versions in more detail and found exactly what you said: No stack frames in XE, but stack frames in XE3. Unfortunately my XE2 installation seems to be broken (it tries to open a codegear url with the word "tampering" in it, so I guess it's the bloody copy protection at work). So thanks for the confirmation. I will update my blog post and the conditional defines.

 

I still don't understand what caused me to assume the change was with Delphi 10.2. I cannot reproduce that now.

Edited June 5 by dummzeuch

[Remy Lebeau 1330]

50 minutes ago, dummzeuch said:

Good to know. I just went through several versions in more detail and found exactly what you said: No stack frames in XE, but stack frames in XE3.

It's a shame they didn't simply update the AbstractErrorProc to accept the ReturnAddress as a parameter so it could be passed to the raise statement, eg:

procedure _AbstractError;
begin
  if Assigned(AbstractErrorProc) then
    AbstractErrorProc(ReturnAddress); // <--
  RunErrorAt(210, ReturnAddress);
end;

...

procedure AbstractErrorHandler(AExceptAddr: Pointer);
begin
  raise EAbstractError.CreateRes(@SAbstractError) at AExceptAddr;
                                                //^^^^^^^^^^^^^^
end;

Like they did with RunErrorAt() in XE2.  And also, as can be seen in other areas of System.pas, like the various _UnhandledException() implementations.

Edited June 5 by Remy Lebeau

[Stefan Glienke 1974]

3 hours ago, Remy Lebeau said:

Which is why it is now subject to the compiler's setting for stack frames when compiling System.pas.

No, you can even see the stackframe settings in System.pas because it explicitly specifies them at the very beginning. Furthermore, even without explicitly enabling them, the RTL is being compiled with $W-

It's a compiler implementation that it enables stackframe for any function that uses ReturnAddress - you can double-check that for yourself by compiling the following code and looking at the disassembly for Foo:

 

{$STACKFRAMES OFF}

procedure Bar(p: Pointer);
begin
end;

procedure Foo;
begin
  Bar(returnAddress);
end;

begin
  Foo;
end.

I know this because I was using ReturnAddress in my code and it behaved wrong in XE where it was implemented explicitly as I linked in my post above but without stackframe, this returned a wrong address.

This is why I explicitly enable stackframes for the code that uses this function in XE - see https://bitbucket.org/sglienke/spring4d/src/2dbce92195d699d51fc99dd226c4698748ec8ef9/Source/Base/Spring.pas#lines-3474

Edited June 5 by Stefan Glienke

[Remy Lebeau 1330]

2 hours ago, Stefan Glienke said:

I know this because I was using ReturnAddress in my code and it behaved wrong in XE where it was implemented explicitly as I linked in my post above but without stackframe, this returned a wrong address.

In Indy, I use the return address in just 1 piece of code - the IndyRaiseOuterException() function.  For XE2+, I use System.ReturnAddress(), and in pre-XE2 versions I use an inner function like how Delphi's SysUtils.Abort() does in XE:

      {$I IdStackFramesOff.inc}
procedure IndyRaiseOuterException(AOuterException: Exception);
  procedure RaiseE(E: Exception; ReturnAddr: Pointer);
  begin
    raise E at ReturnAddr;
  end;
asm
  // AOuterException is already in EAX...
  // MOV EAX, AOuterException
  MOV EDX, [ESP]
  JMP RaiseE
end;
      {$I IdStackFramesOn.inc}