Ping-pong between two Application.ProcessMessages

[ErikT 1]

This one might generate some frowns...

 

I've used Application.ProcessMessages quite often, for several reasons. Most of my applications communicate with an external device via a serial port, USB or Ethernet. Each transaction can take some time, and since it is often crucial that my application gets a response before moving on, I include a while loop (with a timeout). In this loop, apart from checking if data has been received, I make sure to make an Application.ProcessMessages call, so the rest of my application isn't blocked. In the past, this has worked fine.

 

Why this method? Because I think multiple threads are way too difficult to work with, so I don't use them unless it is absolutely necessary. And, as I wrote, it works fine. Until now, that is.

 

In my current application, I communicate almost continuously with the external device, to get some monitoring values, status bits and such from the device. This is run by a timer, as this is the only way I can figure out how to make something run seemingly continuously. A bit like the while (1) loop in a C program for a microcontroller.

Apart from the timer based "main loop", I have a few buttons that can be used for sending various commands to the device. These access the same physical port as the "main loop", so I have some boolean variables that are set and cleared, so one process can wait for the other to finish.

 

Now for the juicy part.

 

I have had problems with the application seemingly stalling once in a while, and couldn't figure out why. When this happens, debug stepping just keeps looping through a limited number of instructions in the CPU tab. So for quite some time, I didn't know which part of my own code started the problem.

I also noticed that the application wasn't actually completely stalled, as there was a separate timer that kept feeding data into some graphs. So it was clearly only certain parts that didn't work.

 

Now I added some global debug variables, and added lines in the code, giving the variables different values. Such as

(...)
    repeat
      ButtonDebugValue := 2;
      while (CommBusy or CommCycle) do 
      begin
        ButtonDebugValue := 3;
        Application.ProcessMessages;
        ButtonDebugValue := 4;
(...)

I did that in the suspected procedures, and made sure to place some before and after Application.ProcessMessages, as I had become particularly suspicious about this. As it turned out, two events (OnTimer and OnClick) were both stuck in an Application.ProcessMessages procedure. At the same time. Not even rolling around in the loop, but just stuck in the procedure.

 

My theory: Application.ProcessMessages messes up if it is called more than once at a time. What I mean is: Process C (the button click) is waiting for process B (the timer), which is waiting for process A (comms receive). When the Application.ProcessMessages call in one event allows a different Application.ProcessMessages call in another event, things go south. The application doesn't stall as such, but those two Application.ProcessMessages never get any further, They are just blocked.

 

What do you think? Is this theory plausible?

 

I am not in need of help to fix this – I would just like to know if I am right or wrong. And maybe let this act as a warning against frivolous use of Application.ProcessMessages.

[Uwe Raabe 2064]

That is just like Application.ProcessMessages works: It processes the messages in the queue. If one of those messages calls Application.ProcessMessages in a loop, the outer Application.ProcessMessages will only get control back when that inner loop ends and the event call returns. 

 

IMHO, you can safely remove the frivolous in your last statement.

[ErikT 1]

Of course! If the calls happen in the "wrong" order, so process B's Application.ProcessMessages call ends up waiting for process A to end, then they are well and surely stuck there. Thanks!

 

Often, the only feasible way around using an Application.ProcessMessages call is to use multiple threads. And, to quote "dummzeuch" in a blog entry, that opens another can of worms.

https://blog.dummzeuch.de/2018/09/29/calling-application-processmessages-in-a-delphi-program/

[Uwe Raabe 2064]

26 minutes ago, ErikT said:

Often, the only feasible way around using an Application.ProcessMessages call is to use multiple threads.

It just looks like the easiest to implement, but most of the time it turns out to be the hardest to get it done right.

 

Another approach is to wrap the code into some TThread.ForceQueue construct. F.i. a loop calling Applicaton.ProcessMessages can be refactored like this:

 

procedure DoAllTheThings;
begin
  DoSomething;
  while DoWeNeedToWait do
    Application.ProcessMessages;
  DoWhatEverIsNecessary;
end;

Refactored:

procedure DoAllTheThings;
begin
  DoSomething;
  DoTheRest;
end;

procedure DoTheRest;
begin
  if DoWeNeedToWait then
    TThread.ForceQueue(nil, DoTheRest)
  else
    DoWhatEverIsNecessary;
end;

All the code is still executed in the main thread, but there is no loop blocking anything.

[Remy Lebeau 1436]

23 minutes ago, Uwe Raabe said:

Another approach is to wrap the code into some TThread.ForceQueue construct...

...

All the code is still executed in the main thread, but there is no loop blocking anything.

It also has the added benefit that it has an optional Delay parameter, if you need a longer sleep between steps, similar to a timer.

[David Heffernan 2353]

Yeah, use threads 

[Rollo62 539]

You could consider a more asynchronous design, for example with TTask, that could avoid the pitfalls of Applicaton.ProcessMessages completely.
The use of immutable objects could make your design even more robust.

 

procedure TMyProcess.RunMyProcess;
begin
  TTask.Run(
    procedure
    var
        LImmutableStatus : TImmutableObject;
        LImmutable       : IImmutableObject;
    begin
      try

        LImmutableStatus := TImmutableObject.Create;

        while ProcessIsRunning do
        begin
          Step1;
          Step2; 

          LImmutable := LImmutableStatus.Clone;
          
          TThread.Synchronize(nil,  // or better TThread.Queue(    if you want to avoid blocking
            procedure
            begin
              { Update User Interface }
              UpdateUIWhenNeeded( LImmutable );

              LImmutable := nil;
            end
          );

          Step3;

        end;
      finally
        TThread.Queue(nil, UpdateUIWhenNeeded );

        LImmutable       := nil;
        LImmutableStatus.Free;

      end;
    end
  );

end;

Just as a rough idea, without checking your specific needs.

[ErikT 1]

On 4/21/2024 at 10:50 AM, Rollo62 said:

You could consider a more asynchronous design, for example with TTask, that could avoid the pitfalls of Applicaton.ProcessMessages completely.
The use of immutable objects could make your design even more robust.

 

I have no idea what an immutable object is. Will need to look into that.

[Rollo62 539]

Immutable objects have their own, non-changeable state and are therefor thread-safe by design
https://docs.oracle.com/javase/tutorial/essential/concurrency/immutable.html

Edited April 24 by Rollo62