Determining why Delphi App Hangs

[Guest]

13 minutes ago, Attila Kovacs said:

well, you have to learn foreign languages then, or look for a native forum, you see me? friend

hi @Attila Kovacs 

 

deep down you love me, only it’s hard for you to admit in public. but i will take the first step: i fight, but i want you by my side.

 

hug (without intimacy hmmmm) 

[Dalija Prasnikar 1396]

2 hours ago, Attila Kovacs said:

@Dalija Prasnikar Could you explain the difference between "putting a db task into a thread" (thread) vs. "application.processmessages" regarding to the GUI?

First, as we all know (I hope) if application cannot process messages, it becomes unresponsive and OS (and user) can kill it, because it may look like it is dead.

 

Application.ProcessMessages pumps the message queue so the OS can see that application is still alive. There are few problems with that approach.

 

Most obvious problem is that such code becomes re-entrant. That can cause some nasty side-effects depending on the code you are executing. 

But main issue is that you are still running slow operation in main thread, and no matter how much you are able to interrupt that execution with calls to Application.ProcessMessages, this will still slow down UI processing and application may respond to user input slowly. Also, this is not just the problem with reacting to input, but all the calls to Application.ProcessMessages will make that long operation running even longer.

 

Next issue that is often overlooked is that some parts of the long running process may not always execute fast enough. Imagine connecting to some web service and retrieving some very small amount of data. In perfect conditions that may take blink of an eye, but if connection is bad for any reason, this operation can block the main thread for much longer. And Application.ProcessMessages will not help you with that.

 

Also, for cross-platform development, this is more important because mobile devices will kill non responsive application much faster. On Android you cannot even freely perform network operations in the context of the main thread because OS will thow exception if you try. Also on android Application.ProcessMessages no longer works and can deadlock the application.

[Stefan Glienke 2002]

You know that annoying person constantly interrupting and asking you the status of some very busy task you are performing? That person is called Application.ProcessMessages

[Attila Kovacs 629]

"Most obvious problem is that such code becomes re-entrant."

Which code? The one starts the blocking loop or the one starts the thread? They have to backed up both. Don't they?

 

"may respond to user input slowly"

Yes. Just like a thread "reacts" slowly, when you want to cancel it and it just won't stop right away because it's too busy to check if there was a stop signal.

 

"but all the calls to Application.ProcessMessages will make that long operation running even longer"

You are right. 2 cores are more than 1. But no, because we have to move a lot of data between the threads. Many times.

And this is not something to ignore. Putting a piece of blocking code into a thread just because the GUI, could be rocket science.

 

"Imagine connecting to some web ..  Application.ProcessMessages will not help you with that."

Yeah, I can only agree with that, and I'm doing the same and yet I did not think of that.

 

I can't tell anything to the android part.

 

[Attila Kovacs 629]

6 minutes ago, Stefan Glienke said:

You know that annoying person constantly interrupting and asking you the status of some very busy task you are performing? That person is called Application.ProcessMessages

Oo yes, and if you giving status reports, you are the ProgressBar 😛

[David Heffernan 2345]

Doesn't seem like ProcessMessages is really relevant here. Nothing is calling ProcessMessages. 

[Anders Melander 1782]

2 hours ago, David Heffernan said:

Doesn't seem like ProcessMessages is really relevant here. Nothing is calling ProcessMessages. 

No but someone suggested doing so and as always that got the ball rolling.

[ewong 3]

13 hours ago, Dany Marmur said:

I, on the other hand, am wondering how things are going for the OP. Did you solve the problem @ewong?

I took up @Dalija Prasnikar's idea of threads and while the UI isn't hanging,  there's a bunch of dependency control issues.  Like in the thread when I disable the controls,  I realized that I can't (by design) navigate a grid which depends on a dataset which had disablecontrol set.

I'm now wondering if it makes sense two have two datasets.  One which is linked to a grid, the other not linked to any control so that the thread can use that non-linked dataset leaving the grid-linked dataset available for use.

 

But as it stands, it's a definite improvement but like all things I don't understand (threads), I'll need to read up on it.

 

Again, Thanks for the help!!  Very much appreciate it.

 

[Vincent Parrett 750]

@ewong You should not, under any circumstances, update vcl controls in a thread other than the main thread. If it's just updating a progress bar, you can do that quite simply by using PostMessage and send a user message to the form

 

https://www.cryer.co.uk/brian/delphi/howto_send_custom_window_message.htm

 

Use PostMessage (non blocking) rather than SendMessage (blocking).

 

Working with threads in Delphi is harder than it needs to be. I recommend taking a look at 

 

https://github.com/gabr42/OmniThreadLibrary

 

I also created a nice wrapper for omnithread that makes running tasks in background threads really simple :

 

https://github.com/VSoftTechnologies/VSoft.Awaitable

 

I've been using this in my projects for a while now (I was already using OmniThread) and it makes things so much simpler!

[Guest]

4 hours ago, ewong said:

I'm now wondering if it makes sense two have two datasets.  One which is linked to a grid, the other not linked to any control so that the thread can use that non-linked dataset leaving the grid-linked dataset available for use.

Check out the TkbmMemTable "AttachedTo" property!

[Dalija Prasnikar 1396]

10 hours ago, Attila Kovacs said:

"Most obvious problem is that such code becomes re-entrant."

Which code? The one starts the blocking loop or the one starts the thread? They have to backed up both. Don't they?

 

The code where you call Application.ProcessMessages

 

procedure TMainForm.BtnClick1(Sender: TObject);
begin
  DoFoo;
  DoBar;
  DoFork;
end;

procedure TMainForm.BtnClick2(Sender: TObject);
begin
  DoFoo;
  Application.ProcessMessages;
  DoBar;
  Application.ProcessMessages;
  DoFork;
end;

procedure TMainForm.BtnClick3(Sender: TObject);
begin
  TThread.CreateAnonymousThread(
    procedure
    begin
      DoFoo;
      DoBar;
      DoFork;
    end).Start;
end;

 

Problem with Application.ProcessMessages is that code that would execute in one block without interruptions, like in first example, can now be interrupted and some other code can start running in the middle of it. For instance you can click same button twice. Or you can close form and destroy it while other code is still running and using it.

 

Now moving in background thread does not solve those issues, because you can still click the button twice and you can close the form while you running something in the background.  

But, there are few things to consider here. First, while code with ApplicationProcessMessages can be interrupted at any point, code in background thread will run serially, one line after the other, just like the code that does not call Application.ProcessMessages. It is far easier to follow the logic (and debug) such code.

 

One of the arguments for Application.ProcessMessages was that it is easy to use, comparing to multithreading. That is true only on the surface, it seems easy and that is why we were all using it. 

So you have beginner developer writing all kind of poor code in event handlers and application becomes unresponsive, so he is advised to use ApplicationProcessMessages, and he sprinkles his code with those and application might work better (seemingly).  In the reality, he learned nothing, his code is really bad, and sooner or later when some user clicks something unexpected everything will fall apart.

 

When you have to move code to the background thread, you need to think more about it and you need to isolate it better. Mere process of isolating functionality will make that code better. Now, you can still have developer that will just move the whole thing to the background without doing any of that, but that code will not seemingly work (that rarely happens) and it will not work properly, which will hopefully trigger the developer to ask for help and learn how to do that properly.

 

10 hours ago, Attila Kovacs said:

"may respond to user input slowly"

Yes. Just like a thread "reacts" slowly, when you want to cancel it and it just won't stop right away because it's too busy to check if there was a stop signal.

There is a difference. If the UI reacts slowly that means when you drag the window and it will not drag smoothly, when you click the button you may not get immediate response, so you will click it again. Background thread does not block the UI so it does not matter how long it takes to cancel the operation. You can write your code in such manner that when action is canceled, user can immediately continue doing something else.

 

10 hours ago, Attila Kovacs said:

"but all the calls to Application.ProcessMessages will make that long operation running even longer"

You are right. 2 cores are more than 1. But no, because we have to move a lot of data between the threads. Many times.

And this is not something to ignore. Putting a piece of blocking code into a thread just because the GUI, could be rocket science.

You don't have to move data between threads - data is not contained within threads (unless you specifically create thread local data), you just need to prevent multiple threads changing the same data simultaneously.

 

[Fr0sT.Brutal 900]

15 minutes ago, Dalija Prasnikar said:

Problem with Application.ProcessMessages is that code that would execute in one block without interruptions, like in first example, can now be interrupted and some other code can start running in the middle of it. For instance you can click same button twice. Or you can close form and destroy it while other code is still running and using it.

Showing a non-closable modal form eliminates such issues

[Dalija Prasnikar 1396]

5 minutes ago, Fr0sT.Brutal said:

Showing a non-closable modal form eliminates such issues

Yes, it does. Until some timer throws you a curved ball.

 

Yes, it is possible to polish Application.ProcessMessages approach and solve reentrancy issues around it. That still makes it a poor solution because you still have other problems I mentioned before. 

[Attila Kovacs 629]

1 hour ago, Dalija Prasnikar said:

When you have to move code to the background thread

You can't just pick a piece of code and put it in the background thread. You can do it with some code, but not every time.

 

1 hour ago, Dalija Prasnikar said:

You can write your code in such manner that when action is canceled, user can immediately continue doing something else.

No, you can't do that. You can do it in some cases, but not every time.

 

1 hour ago, Dalija Prasnikar said:

You don't have to move data between threads

You know exactly what I meant with that.

 

I'm not against threading but we should keep the things there where they are.

[Dalija Prasnikar 1396]

9 minutes ago, Attila Kovacs said:

You can't just pick a piece of code and put it in the background thread. You can do it with some code, but not every time.

When I said put your code in background thread, I didn't mean literally like run exactly the same code in background thread. But for any code that runs long, Application.ProcessMessages is always the wrong solution. It may be quick fix at some point, but it is still a wrong solution.

First step in writing better code is acknowledging that Application.ProcessMessages is wrong solution. Then and only then one can start going in different direction and start learning and applying better solutions. It can be long process, depending on the existing code you (I don't literally mean you) already have. But the sooner you start, the less work you will have in the future when you start removing Application.ProcessMessages. 

 

If you never start doing that, sooner or later you will encounter code where Application.ProcessMessages will not work as intended and you will not be able to make UI fully responsive when using it. Learning how to use background threads at that point - when you cannot properly take time to experiment and learn will only make it harder.   

 

9 minutes ago, Attila Kovacs said:

No, you can't do that. You can do it in some cases, but not every time.

It depends, but certainly more often than with Application.ProcessMessages approach.

 

[Fr0sT.Brutal 900]

5 hours ago, Dalija Prasnikar said:

Yes, it does. Until some timer throws you a curved ball.

What's wrong with timers in this situation?

 

Well, I'm not a App.PM's fan and I last used it quite long ago but some cases just don't worth much efforts. Anyway if one dislikes App.PM, and if a process could be split, he can use PostMsg

 

procedure TForm.MsgDoNext; message MSG_DONEXT;
begin
  DS.Next;
  if DS.Eof then Exit;
  ..do stuff..
  PostMessage(Self.Handle, MSG_DONEXT, 0, 0);
end;

 

[Dalija Prasnikar 1396]

6 minutes ago, Fr0sT.Brutal said:

What's wrong with timers in this situation?

Modal dialog will prevent user triggering actions, but it will not prevent triggering timers (if you have one) from causing possible unintended actions running at wrong time.

 

 

[Wil van Antwerpen 25]

12 hours ago, ewong said:

I took up @Dalija Prasnikar's idea of threads and while the UI isn't hanging,  there's a bunch of dependency control issues.  Like in the thread when I disable the controls,  I realized that I can't (by design) navigate a grid which depends on a dataset which had disablecontrol set.

I'm now wondering if it makes sense two have two datasets.  One which is linked to a grid, the other not linked to any control so that the thread can use that non-linked dataset leaving the grid-linked dataset available for use.

 

But as it stands, it's a definite improvement but like all things I don't understand (threads), I'll need to read up on it.

 

Again, Thanks for the help!!  Very much appreciate it.

 

Just buy Dalija's book "Delphi Event-based and Asynchronous Programming"

It is very good in explaining all the pitfalls and a real bargain for all the knowledge that it carries.

[Attila Kovacs 629]

1 hour ago, Dalija Prasnikar said:

Modal dialog will prevent user triggering actions, but it will not prevent triggering timers (if you have one) from causing possible unintended actions running at wrong time.

 

I need some clarifications here too, as I'm not using timers anywhere, maybe my 3rd parties does, I don't know, but what is "wrong time" for a timer?

I can't remember having seen a property like OnWrongTime or OnRightTime.

Is it possible that you are referring here to "wrong code" instead?

[Guest]

C'mon peeps - keep to the OP topic. No one posting of late actually needs the help regarding message pumping/threads. You are just having a FUD IMHO.

Edited March 26, 2021 by Guest
peepto plural

[Dalija Prasnikar 1396]

17 hours ago, Attila Kovacs said:

I need some clarifications here too, as I'm not using timers anywhere, maybe my 3rd parties does, I don't know, but what is "wrong time" for a timer?

I can't remember having seen a property like OnWrongTime or OnRightTime.

Is it possible that you are referring here to "wrong code" instead?

I meant wrong time. Let me explain.

 

procedure TMainForm.OnTimer(Sender: TObject);
begin
  ChangeFooBarForkState;
end;

procedure TMainForm.BtnClick1(Sender: TObject);
begin
  DoFoo;
  DoBar;
  DoFork;
end;

procedure TMainForm.BtnClick2(Sender: TObject);
begin
  DoFoo;
  Application.ProcessMessages;
  DoBar;
  Application.ProcessMessages;
  DoFork;
end;

Let's say that DoFoo, DoBar and DoFork depend on some state that is changed on timer with ChangeFooBarForkState and if you call DoFoo, DoBar and DoFork in sequence with different state you will get wrong result.

 

In BtnClick1 without ProcessMessages, those procedures will run serially without interruption and with consistent state, because OnTimer event cannnot run in the middle, but in BtnClick2 timer can be triggered and can mess up that state.

 

Now, this is a bit contrived example and you can say that you can fix such code to work properly with timer, but the point is that just like thread safety can sometimes be tricky because code may work properly most of the time, ProcessMessages can also be tricky to do right because you have to think about all the potential messages that can be processed and it can be easy to overlook some and you may have random bad behavior that is hard to reproduce. 

 

Now, this timer example is based on real life scenario someone had somewhere, where Application.ProcessMessages was the problem, I just don't remember where I read about it or what was exact code in question.

[Fr0sT.Brutal 900]

On 3/26/2021 at 4:45 PM, Dalija Prasnikar said:

Modal dialog will prevent user triggering actions, but it will not prevent triggering timers (if you have one) from causing possible unintended actions running at wrong time.

Sure! But resolvable. I, OTOH, like the fact that modal dialogs won't block internal message queue. In my apps Windows' messaging is used massively for communication between threads and I can open some child dialog without stopping the main workflow

[Guest]

10 minutes ago, Fr0sT.Brutal said:

I, OTOH, like the fact that modal dialogs won't block internal message queue

Yes! A good example is RTC message driven Client with multithreading. Quite brilliant.

[aehimself 396]

On 3/25/2021 at 8:56 PM, Dalija Prasnikar said:

Also, this is not just the problem with reacting to input, but all the calls to Application.ProcessMessages will make that long operation running even longer.

This is the exact thing I experienced. I took the shortcut when I converted my app to use background threads for operations. Instead of

Connection.Connect;
DoSomeStuff;

I started a thread and did...

StartConnectionInBackgroundThread;
While Not thread.Finished Do
Begin
//  Sleep(10);
  Application.ProcessMessages;
End;
DoSomeStuff;

All was fine until only one code like this ran. In the moment two tried to connect simultaneously (not mentioning a couple of occasional deadlocks) connections took 5-10 times more than they normally would, if they connect by themselves. Also, UI was responsive, but sluggish.

 

For 20+ builds now I'm in the process to get rid of this shortcut I thought was a good idea before. And while it did what I wanted to achieve on the surface - it did backfire on the long run. For 2+ weeks I'm making changes what the end user might not even realize, and these changes affect 20-30 source files per piece.

 

I completely agree with @Dalija Prasnikar and some others on this matter - I wish I didn't know Application.ProcessMessages exists in the first place 🙂 

[Anders Melander 1782]

8 minutes ago, aehimself said:

I wish I didn't know Application.ProcessMessages exists in the first place 🙂 

You're in good company; We've all been there