Is it a problem if I create more threads than host CPU has ?

[William23668 8]

Hi

 

If I created for example 8 threads to optimize performance but the host machine has CPU with 4 threads only will this be a problem ?

 

Thanks

[stijnsanders 35]

No, not al all. Remember a few decades ago, when most personal computers had only one processor? We had multiple threads then, the operating system cycles them for you nicely, even over the several processors if your system has more.

If you're running really intensive work in the threads, you may cause slowdowns by having more threads than processors (it's interesting to learn about thread affinity as well), but as soon as you're doing I/O from these threads (and the system handles device calls on your threads) you may get an increase of total performance, because the system can let more threads work while some threads are 'waiting'.

There's more to the story when you're using completion ports, or do overlapped IO calls... But only remotely related to your question about threads.

Edited October 27, 2023 by stijnsanders

[Dave Novo 48]

If all threads are fully utilizing the CPU you will more than likely cause slowdowns. The CPU will force one thread to stop, have to push its stack and pop the stack of the new thread, etc etc.  As indicated above, for any process where each thread is waiting a lot (not just IO but really anything) you can happily create as many threads as you like (within reason).

[miab 25]

Normally, after startup, there are already about 1500 threads running in Windows itself

Edited October 27, 2023 by miab

[David Heffernan 2300]

Depends what the threads are doing 

[William23668 8]

17 minutes ago, David Heffernan said:

Depends what the threads are doing 

Looping through very long list of arrays for read and write

[David Heffernan 2300]

10 hours ago, William23668 said:

Looping through very long list of arrays for read and write

There won't be a problem, but it just won't be efficient to use more threads than processors. 

 

My question for you is why you want to do that? 

[William23668 8]

6 hours ago, David Heffernan said:

My question for you is why you want to do that? 

get results faster

[David Heffernan 2300]

1 hour ago, William23668 said:

get results faster

Just use thousands of threads and get results yesterday, simples.

 

No, if you use too many threads then your will get results slower. How do you even think computers work?!!

[Dalija Prasnikar 1341]

3 hours ago, William23668 said:

get results faster

CPU is a finite resource. You will not get more of it, by throwing in more threads. If there are more threads than CPU cores, those threads will compete with each other for CPU time, and switching between threads also costs some CPU time. So after you overload the CPU, more threads you add the slower it will work.

 

Only if threads are doing some I/O bound work, then you can have more such threads than CPU cores and possibly whatever you are doing can finish in less time. In other words if the thread spends most of the time waiting for some slow I/O operation, then such thread will not fully utilize CPU core and other threads can do useful work in the meantime on that core. However, even in I/O bound operations, there are limitations and if those threads are competing for the same resources, then again, the whole process will run slower.

 

Imagine that CPU core is a shovel and threads are workers. In CPU bound work workers need to dig some ground. If you have same amount of shovels as workers than each worker will use the shovel all the time without having to give it up and it will be able to do the digging at full speed. If you have more workers than shovels, then workers will compete for that shovel. And it will not be in a way that some workers will use the shovel all the time and others will do nothing, but each worker will get very short time to use the shovel, maybe only taking one or two loads, and then it will have to give the shovel to the other worker that is waiting. But transferring shovel from one worker to another takes time, and at the end everything will run slower than if you have only one worker per shovel.

 

On the other hand, if the worker also needs to use a pickaxe, then while worker is using the pickaxe, he does not need the shovel, and someone else can use it until the worker is done with the pickaxe. This is example of I/O bound work, where pickaxe is some I/O resource (network, disk...). Again, if you have more workers that compete for the pickaxes, the whole thing will work slower than single worker using that pickaxe.

 

Now, this is simplified example. When it comes to your application running, it is not just your application that uses the CPU cores, but OS and other processes and applications are also using them, so you also need to take those into account. Now finding the right balance for the actual work can be hard and will depend on other parameters, and usually you don't have to optimize that much. But in simple terms, you definitely don't want to have more threads working at the same time than CPU cores, if those threads are doing CPU bound work. Commonly using as much threads as CPU cores or one less, will do - you can experiment with that, but again this is something worth pursuing for most applications. If you are writing something that will run on specific hardware then you can more easily optimize, but if you need to make it work across different ones, then good optimization for one may be a bad optimization for another.

[William23668 8]

@Dalija Prasnik

Yes the arrays I use affect other operations in storage unites so I am already getting better performance and I wont create 1000 threads of course. 

[William23668 8]

3 hours ago, David Heffernan said:

Just use thousands of threads and get results yesterday, simples.

 

No, if you use too many threads then your will get results slower. How do you even think computers work?!!

You dont know my exact needs so go educate yourself how computer work and keep st upid comments to yourself 

[David Heffernan 2300]

22 hours ago, William23668 said:

You dont know my exact needs

Why would that be? Would it be because you asked a technical question without offering any details.

 

Using more threads than processors can be effective when some of the tasks aren't CPU bound. A good example would be writing to or ready from disk. 

 

Accessing memory as the task I would not expect to be amenable to using more threads than processors. Usually computers have greater peak CPU throughput than memory throughout. So if the work is purely memory access then I'd expect CPUs to be idle. Because the memory access should be the bottleneck. 

 

You are quite right when you point out that nobody can advise you accurately without knowing your precise needs. 

Edited October 29, 2023 by David Heffernan