GExperts supports even more laziness

[dummzeuch 1505]

GExperts supports even more laziness

I got into programming because I am lazy, I’d rather have the computer do the boring work than doing it myself. So it’s no wonder that I always liked Delphi and GExperts because both support my laziness.

Today I added yet another feature to save me a key stroke or mouse click: The “Close Exception Notification” expert.

You probably have encountered those libraries that always raise the same exception class on errors, or even worse, raise the generic Exception class instead of bothering to declare their own. And if I should guess, you have probably been guilty of doing that yourself (as I definitely have).

Why is that a problem, you ask?

Because you can’t easily ignore these exceptions during debugging. You always get this “Debugger Exception Notification” dialog, have to read it and decide whether you want to break and look at the code or ignore the exception.

 

Read on in the blog post.

[Stefan Glienke 2002]

Hooking into the code that displays this dialog would probably do the job but is probably not trivial.

 

I've had this request long ago when I started TestInsight because if you run your unit tests under the debugger and you obviously also have tests that check if the correct exceptions occur if you do something bad this will result in a lot of exceptions cought by the debugger. In that scenario however it would only not show those exceptions that you expected in your unit test but break on all others which makes it a bit more complex than just "clicking the dialog away"

[Mahdi Safsafi 225]

Nice job !

Quote

Unfortunately since this expert cannot directly modify the IDE code and I am not aware of any Open Tools API access to the exception notification, it has to hack the dialog. This results in the dialog being displayed for a short time before the expert “presses” the button for you. This is certainly not ideal but still better than not having it.

EDIT:

Sorry I didn't pay attention to "cannot directly modify IDE code" So Vectored Exception Handler wouldn't help.

Edited July 4, 2020 by Mahdi Safsafi

[dummzeuch 1505]

4 hours ago, Stefan Glienke said:

Hooking into the code that displays this dialog would probably do the job but is probably not trivial.

Yeah, I should have written "not trivial" that rather than "cannot". I basically did it this way because it was easy. Unfortunately I'm not as good at this as Andreas Hausladen. He would probably have just hooked the code an be done with it.

[dummzeuch 1505]

4 hours ago, Mahdi Safsafi said:

Nice job !

EDIT:

Sorry I didn't pay attention to "cannot directly modify IDE code" So Vectored Exception Handler wouldn't help.

 

Quote

Vectored handlers are called in the order that they were added, after the debugger gets a first chance notification, but before the system begins unwinding the stack.

So this is exactly that case where it wouldn't work. But thanks for the link.

[Stefan Glienke 2002]

6 minutes ago, dummzeuch said:

Yeah, I should have written "not trivial" that rather than "cannot". I basically did it this way because it was easy. Unfortunately I'm not as good at this as Andreas Hausladen. He would probably have just hooked the code an be done with it.

Run the IDE under the debugger - make it show that dialog - pause it and look into the call stack (need to look through several threads to find the correct one) - and eventually you will find it.

Then simply hook that call and let it run through your code first. The tricky part is just doing this reliably for all supported versions 

[dummzeuch 1505]

2 hours ago, Stefan Glienke said:

Run the IDE under the debugger - make it show that dialog - pause it and look into the call stack (need to look through several threads to find the correct one) - and eventually you will find it.

Then simply hook that call and let it run through your code first. The tricky part is just doing this reliably for all supported versions 

That sounds a lot easier than it is. Especially since the last assembler code that i really understood was 68000.

[Mahdi Safsafi 225]

3 hours ago, Stefan Glienke said:

Run the IDE under the debugger - make it show that dialog - pause it and look into the call stack (need to look through several threads to find the correct one) - and eventually you will find it.

Then simply hook that call and let it run through your code first. The tricky part is just doing this reliably for all supported versions 

 

Here is what I found :

procedure ShowException(Obj: TObject);                        // @Debug@TDebugger@ShowException$qqrv
procedure GetExceptionMessage (Obj: TObject; var Msg: string) // @Debug@TDebugger@GetExceptionMessage$qqrv

ShowException is the one that shows the dialog. GetExceptionMessage is used to get exception message.

I installed a hook on ShowException and was able to pass the dialog but the IDE debugger still break. Possibly the break is the default action.  

procedure ShowException(Obj: TObject);
var
  Msg: string;
begin
  GetExceptionMessage(Obj, Msg);
  ShowMessage(Msg);
  // TrampolineShowException(Obj);
end;

 

[Stefan Glienke 2002]

@Mahdi Safsafi Oh, yes that's probably inside the debugger that reacts to https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-debug_event - so one needs to probably go a little deeper there 😉

[Ian Branch 127]

Oh Dear!

D2007, D10.3.3 & D10.4.  GExperts svn rev 3201.

Get similar as the Attached error on all three Delphis when building.  The D2007 has a slightly different error report.

Happens on either an svn update or a fresh full svn pull.

 

Hmmm.  Is Delphis the plural of Delphi??  Or is it Delphies??

 

Regards,

Ian

[Mahdi Safsafi 225]

@Stefan Glienke @dummzeuch
I made some debugging and conclude to three place where a hook could be used :

- ShowException : this is the top-level function that call DoShowException->ReportExceptionNotification.

- DoShowException : this is called by ShowException and returns False(break) and True(continue). When returning continue, a call to Run is necessary in order to continue running the application.

- ReportExceptionNotification: this is the best place (no need to call run) but unfortunately it inits internally a list that is used later. without calling the original method an AV would be thrown.

 

type
  TShowException = procedure(Obj: TObject);
  TDoShowException = function(Obj: TObject): Integer;
  TGetExceptionMessage = procedure(Obj: TObject; var Msg: string);
  TReportExceptionNotification = function(Msg: Pointer; Unkown: ShortInt; Unkown2: Pointer): ShortInt;

var
  TrampolineShowException: TShowException;
  TrampolineDoShowException: TDoShowException;
  TrampolineReportExceptionNotification: TReportExceptionNotification;
  GetExceptionMessage: TGetExceptionMessage;
  Hooked: Boolean = false;

procedure ShowExceptionHooked(Obj: TObject);
var
  Msg: string;
begin
  {
    this is the toplevel function.
    calls  DoShowException().
  }
  GetExceptionMessage(Obj, Msg);
  ShowMessage(Msg);
  // TrampolineShowException(Obj);
end;

function DoShowExceptionHooked(Obj: TObject): Integer;
var
  Msg: string;
  MsgResult: Cardinal;
begin
  { DoShowException calls ReportExceptionNotificationHooked }
  GetExceptionMessage(Obj, Msg);
  MsgResult := MessageBox(0, 'Yes=continue No=Break Cancel=OriginalDialog', PChar(Msg), MB_YESNOCANCEL);
  case MsgResult of
    IDYES:
      begin
        Result := 1; // continue.
        // need to call Run();
      end;
    IDNO:
      begin
        Result := 0; // break.
      end;
    IDCANCEL:
      begin
        Result := TrampolineDoShowException(Obj); // original dialog.
      end;
  end;
end;

function ReportExceptionNotificationHooked(Msg: Pointer; Unkown: ShortInt; Unkown2: TObject): ShortInt;
var
  MsgResult: Cardinal;
begin
  {
    this is the best place where to make a patch ... however internally, it inits a list
    if you don't call original function, an AV (list) will be thrown.
  }

  // type of Msg = string.

  MsgResult := MessageBox(0, 'Yes=continue No=Break Cancel=OriginalDialog', '', MB_YESNOCANCEL);
  case MsgResult of
    IDYES:
      begin
        Result := 0; // continue.
        // no need to call Run().
      end;
    IDNO:
      begin
        Result := 1; // break.
      end;
    IDCANCEL:
      begin
        Result := TrampolineReportExceptionNotification(Msg, Unkown, Unkown2); // original dialog.
      end;
  end;
end;

procedure InstallHook();
var
  ShowExceptionPtr: Pointer;
  DoShowExceptionPtr: Pointer;
  ReportExceptionNotificationPtr: Pointer;
begin
  if Hooked then
    exit;

  ShowExceptionPtr := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'), '@Debug@TDebugger@ShowException$qqrv');
  DoShowExceptionPtr := GetProcAddress(GetModuleHandle('win32debugide260.bpl'), '@Win32debug@TNativeDebugger@DoShowException$qqrv');
  @GetExceptionMessage := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'), '@Debug@TDebugger@GetExceptionMessage$qqrv');

  ReportExceptionNotificationPtr := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'),
    '@Exceptionnotificationdialog@ReportExceptionNotification$qqrx20System@UnicodeString83System@%Set$56Exceptionnotificationdialog@TExceptionNotificationOptiont1$i0$t1$i2$%r83System@%Set$56Exceptionnotificationdialog@TExceptionNotificaTY6J18n0Wfo24G1hyGAqTA');

  assert(ReportExceptionNotificationPtr <> nil);
  if Assigned(DoShowExceptionPtr) then
  begin
    @TrampolineDoShowException := InterceptCreate(DoShowExceptionPtr, @DoShowExceptionHooked);
    Hooked := true;
    ShowMessage('hook is installed');
  end;
end;

procedure RemoveHook();
begin
  if Hooked then
    InterceptRemove(@TrampolineDoShowException);
end;

 

[Der schöne Günther 316]

Can I make a suggestion? I have so often tripped over ignoring an exception and then forgetting about it, the exception being ignored even days later. Would it be possible to add something like "scope" for these rules? Something like

1. This debugger session
2. This project
3. Global

[dummzeuch 1505]

6 hours ago, Mahdi Safsafi said:

@Stefan Glienke @dummzeuch
I made some debugging and conclude to three place where a hook could be used :

I'll give that a try, thanks a lot.

[Mahdi Safsafi 225]

Good news guys ! I have found a way to work around continue issue ! Now the example works like a charm 😊

14 hours ago, Stefan Glienke said:

@Mahdi Safsafi Oh, yes that's probably inside the debugger that reacts to https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-debug_event - so one needs to probably go a little deeper there 😉

You opened my mind ... thanks 😉

There is an internal function called PostDebugMessage that do the trick.

type
  TShowException = function(Obj: TObject): Boolean;
  TDoShowException = function(Obj: TObject): Boolean;
  TGetExceptionMessage = procedure(Obj: TObject; var Msg: string);
  TReportExceptionNotification = function(Msg: Pointer; Unkown: ShortInt; Unkown2: Pointer): ShortInt;
  TPostDebugMessage = procedure(Obj: TObject; Msg: ShortInt; Param: Pointer);

var
  TrampolineShowException: TShowException;
  TrampolineDoShowException: TDoShowException;
  TrampolineReportExceptionNotification: TReportExceptionNotification;
  GetExceptionMessage: TGetExceptionMessage;
  PostDebugMessage: TPostDebugMessage;
  Hooked: Boolean = False;

function GetParam(Obj: Pointer): Pointer;
asm
  mov eax, [eax + $40]
end;

function DoShowExceptionHooked(Obj: TObject): Boolean;
var
  Msg: string;
  MsgResult: Cardinal;
  P: PByte;
begin
  GetExceptionMessage(Obj, Msg);
  MsgResult := MessageBox(0, 'Yes=continue No=Break Cancel=OriginalDialog', PChar(Msg), MB_YESNOCANCEL);
  case MsgResult of
    IDYES:
      begin
        { continue. }
        Result := True;
        P := GetParam(Obj);
        P[$A1] := 1; // resume = true.
        PostDebugMessage(Obj, 1, P); // this will resume the app.
      end;
    IDNO:
      begin
        { break. }
        Result := False;
      end;
    IDCANCEL:
      begin
        { original dialog. }
        Result := TrampolineDoShowException(Obj);
      end;
  end;
end;

procedure InstallHook();
var
  ShowExceptionPtr: Pointer;
  DoShowExceptionPtr: Pointer;
  ReportExceptionNotificationPtr: Pointer;
  PostDebugMessagePtr: Pointer;
begin
  if not Hooked then
  begin
    ShowExceptionPtr := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'), '@Debug@TDebugger@ShowException$qqrv');
    DoShowExceptionPtr := GetProcAddress(GetModuleHandle('win32debugide260.bpl'), '@Win32debug@TNativeDebugger@DoShowException$qqrv');
    ReportExceptionNotificationPtr := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'),
      '@Exceptionnotificationdialog@ReportExceptionNotification$qqrx20System@UnicodeString83System@%Set$56Exceptionnotificationdialog@TExceptionNotificationOptiont1$i0$t1$i2$%r83System@%Set$56Exceptionnotificationdialog@TExceptionNotificaTY6J18n0Wfo24G1hyGAqTA');

    @GetExceptionMessage := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'), '@Debug@TDebugger@GetExceptionMessage$qqrv');
    @PostDebugMessage := GetProcAddress(GetModuleHandle('dbkdebugide260.bpl'), '@Debug@TDebugger@PostDebugMessage$qqr15Debug@TDebugMsgpv');

    Assert(Assigned(DoShowExceptionPtr));
    Assert(Assigned(@GetExceptionMessage));
    Assert(Assigned(@PostDebugMessage));

    @TrampolineDoShowException := InterceptCreate(DoShowExceptionPtr, @DoShowExceptionHooked);
    Hooked := True;
    ShowMessage('hook was installed.');
  end;
end;

procedure RemoveHook();
begin
  if Hooked then
  begin
    InterceptRemove(@TrampolineDoShowException);
    Hooked := False;
  end;
end;

N.B: The signature for ShowException was corrected as I found that it is a function and not a procedure.  

@dummzeuch All what we need now is your magic touch ! Perhaps you can make a custom exception dialog that replace the old one. And we would be able then to ignore all exceptions with same message only once instead of going to expert-settings and use regex.  

[dummzeuch 1505]

I have been working on this all day. First go get rid of having to use DDetour, because it doesn't compile with Delphi 2007 and older (Haven't checked 2006 and 2005), then to find out where the functions are that I need to hook in older Delphi versions. Now I was in the process of getting it all together, and there you come up with a solution the requires me to start from the beginning again. 😐

 

(Not that I'm complaining, I'm just a bit frustrated by now. I highly value your input.)

[dummzeuch 1505]

var

  P: PByte

[...]

   P[$A1] := 1; // resume = true.

 

This doesn't compile. Am I right to assume that it sets PByte(P + $A1) to 1 ?

 

Edit: Seems to work, so the assumption probably was right.

Edited July 5, 2020 by dummzeuch

[dummzeuch 1505]

I also need the exception class, but that's easy: @Debug@TDebugger@GetExceptionName$qqrv

 

 

[Mahdi Safsafi 225]

37 minutes ago, dummzeuch said:

I have been working on this all day. First go get rid of having to use DDetour, because it doesn't compile with Delphi 2007 and older (Haven't checked 2006 and 2005), then to find out where the functions are that I need to hook in older Delphi versions. Now I was in the process of getting it all together, and there you come up with a solution the requires me to start from the beginning again. 😐

 

(Not that I'm complaining, I'm just a bit frustrated by now. I highly value your input.)

I'm very sorry man. I loved to give you the full solution if I was able but debugging/reversing Delphi IDE is a nightmare. A lot of assembly code must be reviewed/tracked/commented, following a bunch of stack-call to find the desired method, and the Delphi IDE keeps throwing many exceptions under the debugger that forces me to make a restart. RE is a hard domain and sometime you take weeks/month to fully reverse something.

 

The above solution I gave is the native one. Delphi IDE uses it to resume application. So I believe its your best option.

Sorry for DDetours, I got many features for it but never find a time 😥

 P[$A1] := 1; // resume = true.
This doesn't compile. Am I right to assume that it sets PByte(P + $A1) to 1 ?

It compiles perfectly under D10.3.2 so I guess you're using an old version. FWIW, you can use PByte(P + $A1)^ := 1; and you can do the same for GetParam (I used it for exploration and I forgot to replace it).

Quote

I also need the exception class, but that's easy: @Debug@TDebugger@GetExceptionName$qqrv

Yes that's it. Just be careful for its signature :

TGetExceptionName = procedure(Obj: TObject; var Msg: string);

 

[dummzeuch 1505]

37 minutes ago, Mahdi Safsafi said:

Sorry for DDetours, I got many features for it but never find a time

I know this feeling. 😉

 

I found some hooking code I had used before and works fine, probably originally from David Heffernan on StackOverflow.

It doesn't support unhooking though which is a pity but not really a problem.

 

37 minutes ago, Mahdi Safsafi said:

 P[$A1] := 1; // resume = true.
This doesn't compile. Am I right to assume that it sets PByte(P + $A1) to 1 ?

It compiles perfectly under D10.3.2 so I guess you're using an old version.

Yes, Delphi 2007 is the first one I always try. If it works there, it most likely works with all newer versions.

37 minutes ago, Mahdi Safsafi said:

FWIW, you can use PByte(P + $A1)^ := 1; and you can do the same for GetParam (I used it for exploration and I forgot to replace it).

I needed to typecast to integer and back to make it compile:

    PByte(Integer(P) + $A1)^ := 1; // resume = true.

 

(Good thing I don't have to worry about 64 bits as long as the IDE stays 32 bits.)

 

I have got it working now, just not yet quite as polished as I'd like it. Should work for all Delphi versions from 2005 upwards. I'll keep the old code for Delphi 6 and 7.

Edited July 5, 2020 by dummzeuch

[dummzeuch 1505]

Committed in revision #3209, just in case anybody wants to have a look.

 

Thanks a lot @Mahdi Safsafi

I would never have come that far without your help!

 

@Der schöne Günther could you please file a feature request for your suggestion ?

 

[Mahdi Safsafi 225]

26 minutes ago, dummzeuch said:

I found some hooking code I had used before and works fine, probably originally from David Heffernan on StackOverflow.

 It doesn't support unhooking though which is a pity but not really a problem.

The purpose of DDetours v2 is to achieve a correct management of several hooks on the same function. Meaning if two expert are hooking the same function, DDetours guarantees that expert2 will receive the hook when calling NextHook/Trampoline from expert1.

If I have a 3rd party IDE plugin, I wouldn't miss a chance to add a stack trace for the exception dialog. This could be possible together with expert1 only if both expert uses DDetours.

If possible, use DDetours as DLL for older version.

Quote

     PByte(Integer(P) + $A1)^ := 1; // resume = true.

I think that bellow code will work with older version and is more safe :

type
  TArrayOfShortInt = array [Byte] of ShortInt; // bound safe
  PArrayOfShortInt = ^TArrayOfShortInt;

var P: PArrayOfShortInt;
P := PArrayOfShortInt(GetParam(Obj));
P^[$A1] := 1; // safe than pointer arithmetic.

 

Quote

Thanks a lot @Mahdi Safsafi

I would never have come that far without your help!

You're welcome   

[Ian Branch 127]

Hi Thomas,

Rev 3209 Builds in D2007, D10.3.3 & S10.4 OK.

Got the attached message when I run D10.4.

Can't find TFormHotKeysExport.bmp on the PC.

 

Regards,

Ian

 

Edited July 5, 2020 by Ian Branch

[dummzeuch 1505]

Yet another expert that's work in progress and doesn't have an icon yet. I think I should add a default icon for those experts that don't have one. Maybe some "roadworks" sign.

[Ian Branch 127]

Hi Thomas,

Now that's an excellent idea. 🙂

I wasn't being picky, just making you aware.

Regards,

Ian

[dummzeuch 1505]

20 hours ago, Mahdi Safsafi said:

The purpose of DDetours v2 is to achieve a correct management of several hooks on the same function. Meaning if two expert are hooking the same function, DDetours guarantees that expert2 will receive the hook when calling NextHook/Trampoline from expert1.

If I have a 3rd party IDE plugin, I wouldn't miss a chance to add a stack trace for the exception dialog. This could be possible together with expert1 only if both expert uses DDetours.

If possible, use DDetours as DLL for older version.

There are two reasons why I didn't want to use DDetours:

1. I didn't want to add yet another 3rd party library to GExperts. Using 3rd party libraries means that with a new Delphi version I have to either update these libraries myself to support the new version or wait for the maintainer to do it. And even if that wasn't a problem, you never know when such a library drops support for older Delphi versions or maybe just adds changes without keeping older Delphi versions in mind. On top of it they always add complexity and increase executable size and memory requirements.
2. DDetours did not compile with Delphi 2007. Since that's currently my main development version this was the end of it.

I see the advantage of allowing chained hooks, but supporting Delphi 2007 is still more important to me.