multithreading - How to find out the optimal amount of threads?

Question

I'm planning to make a software with lot of peer to peer like network connections. Normally I would create an own thread for every connection to send and receive data, but in this case with 300-500+ connections it would mean continuously creating and destroying a lot of threads which would be a big overhead I guess. And making one thread that handles all the connections sequentially could probably slow down things a little. (I'm not really sure about this.)

The question is: how many threads would be optimal to handle this kind of problems? Would it be possible to calculate it in the software so it can decide itself to create less threads on an old computer with not as much resources and more on new ones?

It's a theoretical question, I wouldn't like to make it implementation or language dependant. However I think a lot of people would advice something like "Just use a ThreadPool, it will handle stuff like that" so let's say it will not be a .NET application. (I'll probably has to use some other parts of the code in an old Delphi project, so the language will be probably Delphi or maybe C++ but it's not decided yet.)

Answer 1

Understanding the performance of your application under load is key, as mentioned before profiling, measurements and re-testing is the way to go.

As a general guide Goetz talks about having

threads = number of CPUs + 1

for CPU bound applications, and

number of CPUs * (1 + wait time / service time)

for IO bound contexts