Many TTasks

[Jud 1]

If you have started more TTasks than you have logical CPUs, does it do them in parallel or does it finish one before starting the next one in the queue?  Example, you have a CPU with 8 logical cores and start 9 TTasks.  Does it finish one of the first 8 before starting #9?

[Dalija Prasnikar 1396]

20 minutes ago, Jud said:

If you have started more TTasks than you have logical CPUs, does it do them in parallel or does it finish one before starting the next one in the queue?  Example, you have a CPU with 8 logical cores and start 9 TTasks.  Does it finish one of the first 8 before starting #9?

Tasks use threads from associated thread pool. If you don't explicitly specify thread pool, default one will be used. Maximum number of  threads each pool can start (use) can be modified for each pool, and default value is number of CPUs * 2. (This value is valid for Delphi 11, it is possible that it was some other value in previous versions)

 

So if you have 8 CPUs and you start 9 tasks, they will start 9 threads running in parallel, unless one of the tasks manages to finish before all 9 tasks are queued to tasks work queue. 

 

You should always assume that tasks will run in parallel, in other words you cannot count on particular task finishing before some other task.

Edited November 16, 2021 by Dalija Prasnikar

[Jud 1]

Thanks, that is what I was wondering.  I know how to wait for ONE task to finish and to wait for ALL tasks to finish.  So for 9 tasks with 8 logical CPUs, I can wait for one to finish before starting the 9th one.  But what if there are, say, 20 tasks, and I don't want to run all so in parallel.  Is there an easy way to keep only 8 running in parallel at the time?

[David Schwartz 426]

15 minutes ago, Jud said:

Is there an easy way to keep only 8 running in parallel at the time?

How do you know there are exactly 8 (or 'n') CPUs?

[David Heffernan 2345]

33 minutes ago, Jud said:

Thanks, that is what I was wondering.  I know how to wait for ONE task to finish and to wait for ALL tasks to finish.  So for 9 tasks with 8 logical CPUs, I can wait for one to finish before starting the 9th one.  But what if there are, say, 20 tasks, and I don't want to run all so in parallel.  Is there an easy way to keep only 8 running in parallel at the time?

Create a thread pool with 8 threads and assign the tasks to that thread pool.

 

Incidentally does anybody know why the default thread pool has more threads than CPUs? 

[David Heffernan 2345]

19 minutes ago, David Schwartz said:

How do you know there are exactly 8 (or 'n') CPUs?

Machines have a fixed and well defined number of processors that can readily be queried. So it's trivial to know what N is. 

[David Schwartz 426]

1 minute ago, David Heffernan said:

Machines have a fixed and well defined number of processors that can readily be queried. So it's trivial to know what N is. 

I was trying to get him to figure that out himself....

[David Heffernan 2345]

6 minutes ago, David Schwartz said:

I was trying to get him to figure that out himself....

The asker already knows this. The question is not how to find N but how to control task scheduling on thread pools. 

[Jud 1]

30 minutes ago, David Schwartz said:

How do you know there are exactly 8 (or 'n') CPUs?

I was just using that as an example.  But I have a routine that tells me the number of logical cores.

[David Schwartz 426]

3 minutes ago, Jud said:

I was just using that as an example.  But I have a routine that tells me the number of logical cores.

That's why I said "(or 'n')"...

 

So, knowing that 'n' <= #cores, you can limit how many active tasks or threads there are to 'n', right? I would think the easiest way to do that is to set the size of the thread pool to 'n'.

 

(Sorry if that's too obvious.)

[Dalija Prasnikar 1396]

17 minutes ago, David Heffernan said:

Create a thread pool with 8 threads and assign the tasks to that thread pool.

 

Incidentally does anybody know why the default thread pool has more threads than CPUs? 

Wild guessing: 

Someone mixed up number of cores with number of logical processors where hyperthreading allows two threads per physical core. CPUCount returns number of logical processors so doubling that is wrong.

On other hand, probably more accurate guess would be that this number is used as some sort of "best" default because not all tasks are CPU bound, and for IO bound tasks more threads than logical CPUs gives better performance. 

Optimizing number of threads is not always easy as it is highly dependent on actual tasks.

[Jud 1]

But I know that the number of tasks is going to be considerably larger than the number of logical CPUs. 

[Dalija Prasnikar 1396]

3 minutes ago, Jud said:

But I know that the number of tasks is going to be considerably larger than the number of logical CPUs. 

Generally, you don't have to worry about that. The pool handles number of threads for you, depending on minimum and maximum number of threads set for particular pool, and how much is system stressed out (busy) at the time. Only if you see some visible performance issues - tasks don't run in expected time frame or you are not utilizing resources as much as you could (you have too many slow IO bound tasks) you should try to optimize minimal and maximal number of threads per particular thread pool.

 

The only next possible thing you need to worry about (again you need to test the real life scenario) is flooding the task queue with extremely large numbers of tasks at once (probably thousands) where growing task queue can pose a problem.

[David Heffernan 2345]

2 hours ago, Jud said:

But I know that the number of tasks is going to be considerably larger than the number of logical CPUs. 

So what? Assuming your tasks for CPU bound then set the thread pool thread count to match the number of CPUs, and let the scheduler do its job.

[Uwe Raabe 2057]

Even if you know that there are 8 threads possible to run in parallel on your machine, it isn't guaranteed that this will actually happen. There can be other tasks from other processes or the OS itself occupying some CPUs which leaves you in the same problem as having more tasks than there are CPUs.

[David Heffernan 2345]

3 minutes ago, Uwe Raabe said:

the same problem as having more tasks than there are CPUs

Just to be pedantic, but more "tasks" than CPUs is not a problem. The problem is when you have more runnable threads than CPUs.