try... finally on Mac

[Brandon Staggs 278]

We are porting a very large and mature Delphi project from Windows to Mac. This project has the try... finally concept pretty well baked-in to recover from catastrophic errors that should not bring the whole application down. Now I am discovering that this approach is not reliable on Mac. To start with, I should have paid more attention to this in the Delphi documentation:

 

Quote

On the macOS 64-bit (Intel) and macOS 64-bit (ARM) platform, structured exception handling (__try/__except) is not available.

https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Differences_between_Windows_and_macOS

 

But in reality, try... finally does work, if the exception originates in the RTL or some other Delphi class. If you raise an exception, finally blocks will execute as expected.

 

The problem is when the operating system is the source of the exception. For example:

 

procedure TForm1.Button_NilClick(Sender: TObject);
begin
    if not TMonitor.TryEnter(FLock) then
        begin
        Log.d('Still locked.');
        exit;
        end
    else
    	TMonitor.Exit(FLock);

    TThread.CreateAnonymousThread(
        procedure
        var
            o: TStringList;
        begin
            Log.d('***Access nil object thread***');
            o := nil;
            TMonitor.Enter(Form1.FLock);
            try
                o.Clear; // raises exception for sure.
            finally
                // Never executed on Mac
                TMonitor.Exit(Form1.FLock);
                Log.d('o.Clear was called. (finally block)');
                end;
        end
    ).Start;
end;

On Windows, the lock is always released in the Finally block, of course.

 

On Mac, it appears that the operating system simply kills the thread. The Log call and the lock release in the finally block is never executed.

 

So, what is the right way to handle this? Obviously we don't want to call methods on an uninitialized object reference, but what pattern can be followed on Mac to make sure a lock is released after running some code that could potentially fail outside of our control?

 

Also, I am concerned about the implications of this with TTask and the thread pool. Last I looked (and I do need to verify this anew), Delphi uses code in finally blocks to handle completion of tasks and passing off to new tasks, etc. If there is an external exception in one of those tasks, how can the worker threads not become corrupt?

Edited February 1, 2023 by Brandon Staggs

[Brandon Staggs 278]

Also, a rant: I feel like a trailblazing pioneer on the frontier of some unexplored land. Searching this issue out has led me to one and only one thread where someone brought this up on Stack Overflow, and that thread was incorrectly marked as a duplicate (neither of the linked answers are relevant to the question in the slightest). The comments on the SO post are a bunch of people piling on about the cause of the exception and not addressing the question that was asked. Am I the only one actually trying to use Delphi to build a non-trivial MacOS application based on an existing code base?

Edited February 1, 2023 by Brandon Staggs

[DelphiMT 5]

I feel for you. I think a big part of the problem is that we lost a lot the developer community when Google shut down Google+ -- there was a pretty active Delphi group there, and while this forum is great, I don't think it has the reach of the old Google+ group. You might try posting on Reddit and Discord as well -- there's got to be someone out there who has solved this problem.

 

I had a similar experience when making an Android app that uses Bluetooth -- much of the Delphi documentation was out-of-date, Stack Overflow wasn't very helpful, and a lot has changed in how you use Bluetooth in recent versions of Android. I do not envy the RAD Studio developers trying to maintain and keep up with 4 different platforms.

 

[Dalija Prasnikar 1396]

2 hours ago, Brandon Staggs said:

So, what is the right way to handle this? Obviously we don't want to call methods on an uninitialized object reference, but what pattern can be followed on Mac to make sure a lock is released after running some code that could potentially fail outside of our control?

This issue happens on all LLVM backed compilers because LLVM is not capable of catching hardware exceptions unless exception happens within another function. 

 

You find more information on my blog post https://dalijap.blogspot.com/2018/10/catch-me-if-you-can.html I never got around to write the sequel, but the implications is that literally anything that all exception handling implicit or explicit is broken in such situations.

 

The solution is that you either wrap your code in separate function that will not have any exception handling within, and then the caller will be able to catch and handle raised exceptions. Another way of solving issues is to avoid code that can trigger such hardware exception and raise Delphi exception if code does not satisfy some requirement as explained in https://docwiki.embarcadero.com/RADStudio/en/Migrating_Delphi_Code_to_Mobile_from_Desktop#Use_a_Function_Call_in_a_try-except_Block_to_Prevent_Uncaught_Hardware_Exceptions

 

So in the context of your example, you should either check whether object is nil before trying to use it (this would actually be general advice, as accessing nil object, depending on the code, on Windows does not guarantee that you will get AV).

 

Note. I don't know what exactly following quote from documentation about macOS means: "structured exception handling (__try/__except) is not available". Linked page talks about hardware exceptions, but I am not sure whether there are some other implications here besides what I said in context of LLVM. At the moment I don't have my development environment set up in a way that would allow me to verify behavior on Mac. 

2 hours ago, Brandon Staggs said:

Also, I am concerned about the implications of this with TTask and the thread pool. Last I looked (and I do need to verify this anew), Delphi uses code in finally blocks to handle completion of tasks and passing off to new tasks, etc. If there is an external exception in one of those tasks, how can the worker threads not become corrupt?

While bugs are always possibility, RTL appropriately handles hardware exceptions in cross-platform code. 

 

2 hours ago, Brandon Staggs said:

Also, a rant: I feel like a trailblazing pioneer on the frontier of some unexplored land. Searching this issue out has led me to one and only one thread where someone brought this up on Stack Overflow, and that thread was incorrectly marked as a duplicate (neither of the linked answers are relevant to the question in the slightest).

I removed wrong duplicates and added appropriate one. 

[Brandon Staggs 278]

51 minutes ago, Dalija Prasnikar said:

I removed wrong duplicates and added appropriate one. 

Thank you. I think I was being too restrictive in my search (always looking for something including "MacOS" or "Mac") and that was keeping your posts from me. Thank you for the details, it is very helpful!

[Brandon Staggs 278]

1 hour ago, Dalija Prasnikar said:

So in the context of your example, you should either check whether object is nil before trying to use it (this would actually be general advice, as accessing nil object, depending on the code, on Windows does not guarantee that you will get AV).

Just to be clear, the nil object method call to Clear was deliberately designed to cause an access violation. I am more concerned about the non-obvious things that happen outside of our control. In fact, my primary concern (not stated) is dealing with access violations inside of library calls; libraries we have to use but to not have control over. And in fact, I don't actually need to worry about that in particular, since calling a function in a dylib that leads to an AV will return to our function and execute the finally block anyway.

 

Again, thanks for your explanation.

[Dalija Prasnikar 1396]

10 hours ago, Brandon Staggs said:

Just to be clear, the nil object method call to Clear was deliberately designed to cause an access violation.

Don't worry, it was evident that this was just an example to show the failure.  

 

Obviously in real code one would not set object to nil and then use if Assigned() to check whether object is nil before using it. But if you have scenario where some function call can return nil object as valid response - for instance finding object that satisfies condition in a collection, and returns nil if the appropriate object is not found, then you have to check for nil before using it. Once you have valid object, calling methods on it, even if they raise exceptions will be caught by surrounding exception handler. 

10 hours ago, Brandon Staggs said:

I am more concerned about the non-obvious things that happen outside of our control. In fact, my primary concern (not stated) is dealing with access violations inside of library calls; libraries we have to use but to not have control over. And in fact, I don't actually need to worry about that in particular, since calling a function in a dylib that leads to an AV will return to our function and execute the finally block anyway.

Again, in such cases you will be calling some library function, and unless you are calling something on nil object this will be fine. The potential issues may arise if the library itself (Delphi ones) is not written with LLVM based compilers in mind, where some exceptions will won't be caught by appropriate exception handlers within the library code. If that happens then the library code has to be fixed, you cannot solve the problem from your code.

[Rollo62 536]

13 hours ago, Brandon Staggs said:

.. I am more concerned about the non-obvious things that happen outside of our control. ...

Since it is said that such exceptions bubble up one level, maybe the naive approach will maybe lead to

var
  P: ^Integer = nil;

procedure G1;
begin
  P^ := 42;
end;

begin

  try
     try
         G1;

     except  //<= Let me bubble up one level
         writeln('Catch:G1 - if you can');
     end;

  except
    writeln('Catch:G1 - pass on a 2nd level wherever needed');
  end;
end.

Not sure if that is flawless for any kind of exceptions, any ideas ?

 

Edited February 2, 2023 by Rollo62

[Dalija Prasnikar 1396]

28 minutes ago, Rollo62 said:

Not sure if that is flawless for any kind of exceptions, any ideas ?

If the G! is inlined then exception will not be caught by second level try..except block. It can only be caunght by exception handler outside the method where exception happens.

 

var
  P: ^Integer = nil;

procedure G1; inline;
begin
  P^ := 42;
end;

procedure TForm1.G2;
begin
  try
     try
       G1;
     except
       Memo1.Lines.Add('Inner');
     end;
  except
    Memo1.Lines.Add('Outer');
  end;
end;

procedure TForm1.Button1Click(Sender: TObject);
begin
  try
    G2;
  except
    Memo1.Lines.Add('Proc');
  end;
end;

In other words, in above code exception will be caught by exception handler around G2 -> Proc

 

If we remove inline directive then exception will be caught by Inner exception handler.

 

I don't know whether LLVM can automatically inline functions that are not explicitly marked for inlining, but I would expect that we shouldn't worry about that as this would break exception handling not only for Delphi, but for other languages, too.

 

[Rollo62 536]

@Dalija Prasnikar

Thanks, my goal was to solve all special, unforseeable OS hardware exception cases by double-nesting, not necessarily the inline issue.
Of course according to all the above discussions, you better avoid certain configurations, like inline, in this situation and try to introduce for example a static _Internal helper method as wrapper,

so to be able to catch all alien OS AVs.

At least I hope so ....

 

Edited February 2, 2023 by Rollo62

[Dalija Prasnikar 1396]

11 minutes ago, Rollo62 said:

Thanks, my goal was to solve all special, unforseeable OS hardware exception cases by double-nesting, not necessarily the inline issue.

Inline behaves as if you have written the code directly within the try..except method. Just like double nesting does not help with the inlined procedure it will not help with any other code that is written directly in try..except block. Only wrapping the code that can cause exception in additional function, procedure or method is the solution.

[Rollo62 536]

1 hour ago, Dalija Prasnikar said:

Only wrapping the code that can cause exception in additional function, procedure or method is the solution.

Sorry, maybe I did not explain my thoughts clear enough or maybe I misunderstand your meaning here, yes I think I'm on the same track with you.

The inline topic was not brough up by me, and I see this only as a side issue for me ( could be easily avoided ).

My though was never to use inlined function of course, only wrapped in a static function.
But for all other cases this construct should be able to catch any mystical OS hardware exception ( where can be a lot ).