David Carmona
Premier Support for Developers
Microsoft Spain
June 2002
Summary: Provides an in-depth look at the thread pool support in the Microsoft .NET Framework, shows why you need a pool and the implementation provided in .NET, and includes a complete reference for its use in your applications. (25 printed pages)
Contents
Introduction
Thread Pool in .NET
Executing Functions on the Pool
Using Timers
Execution Based on Synchronization Objects
Asynchronous I/O Operations
Monitoring the Pool
Deadlocks
About Security
Conclusion
For More Information
Introduction
If you have experience with multithreaded programming in any programming language, you are already familiar with the typical examples of it. Usually, multithreaded programming is associated with user interface-based applications that need to perform a time-consuming operation without affecting the end user. Take any reference book and open it to the chapter dedicated to threads: can you find a multithreaded example that can perform a mathematical calculation running in parallel with your user interface?
It is not my intention that you throw away your books—don't do that! Multithreaded programming is just perfect for user interface-based applications. In fact, the Microsoft® .NET Framework makes multithreaded programming available to any language using Windows Forms, allowing the developer to design very rich interfaces with a better experience for the end user. However, multithreaded programming is not only for user interfaces; there are times that we need more than one execution stream without having any user interface in our application.
Let's use a hardware store client/server application as an example. The clients are the cash registers and the server is an application running on a separate machine in the warehouse. If you think about it, the server application can have no user interface at all. However, what would the implementation be without multithreading?
The server would receive requests from the clients via a channel (HTTP, sockets, files, etc.); it would process them and send a response back to the clients. Figure 1 shows how it would work.
Figure 1. Server application with one thread
The server application has to implement some kind of queue so that no requests are omitted. Figure 1 shows three requests arriving at the same time, but only one can be processed. While the server executes the request, "Decrease stock of monkey wrench," the other two must wait for their turns to be processed in the queue. When the execution of the first request is finished, the second one is next, and so on. This method is commonly used in many existing applications, but it poorly utilizes system resources. Imagine that decreasing the stock requires a modification of a file on disk. While this file is being written, the CPU will not be used even if in the meantime there are requests waiting to be processed. A general indication of these systems is a long response time with low CPU usage, even in stress conditions.
Another strategy used in current systems is to create different threads for each request. When a new request arrives, the server creates a new thread dedicated to the incoming request and it is destroyed when the execution finishes. The following diagram shows this case:
Figure 2. Multithreaded server application
As Figure 2 illustrates, we won't have a low CPU usage now—just the opposite. Even if it is not slow, creating and destroying threads is not optimum. If the operations performed by the thread are not complex, the extra time it takes to create and destroy threads can severely affect the final response time. Another point is the huge impact of these threads in stress conditions. Having all the requests executing at once on different threads would cause the CPU to reach 100% and most of the time would be wasted in context switching, even more than processing the request itself. Typical behaviors in this kind of system are an exponential increment of response time with the number of requests, and a high usage of CPU privileged time (this time can be viewed with the Task Manager and is affected by context switches between threads).
An optimum implementation is based on a hybrid of the previous two illustrations and introduces the concept of thread pool. When a request arrives, the application adds it to an incoming queue. A group of threads retrieves requests from this queue and processes them; as each thread is freed up, another request is executed from the queue. This schema is shown in the following figure:
Figure 3. Server application using a thread pool
In this example, we use a thread pool of two threads. When three requests arrive, they are immediately placed in the queue waiting to be processed; because both threads are free, the first two requests begin to execute. Once the process of any of these requests is finished, the free thread takes the third request and executes it. In this scenario, there is no need to create or destroy threads for each request; the threads are recycled between them. If the implementation of this thread pool is efficient, it will be able to add or remove threads from the pool for best performance. For example, when the pool is executing two requests and the CPU doesn't reach 50% utilization, it means that the executed requests are waiting for events or doing some kind of I/O operation. The pool can detect this situation and increase the number of threads so that more requests can be processed at the same time. In the opposite case when the CPU reaches 100% utilization, the pool decreases the number of threads to get more real CPU time without wasting it in context switches.
Thread Pool in .NET
Based on the above example, it is crucial to have an efficient implementation of the thread pool in enterprise applications. Microsoft realized this in the development of the .NET Framework; included in the heart of the system is an optimum thread pool ready for use.
This pool is not only available to applications that want to use it, but it is also integrated with most of the classes included in the Framework. In addition, there is important functionality of .NET built on the same pool. For example, .NET Remoting uses it to process requests on remote objects.
When a managed application is executed, the runtime offers a pool of threads that will be created the first time the code accesses it. This pool is associated with the physical process where the application is running, an important detail when you are using the functionality available in the .NET infrastructure to run several applications (called application domains) within the same process. If this is the case, one bad application can affect the rest within the same process because they all use the same pool.
You can use the thread pool or retrieve information about it through the class ThreadPool, in the System.Threadingnamespace. If you take a look at this class, you will see that all the members are static and there is no public constructor. This makes sense, because there's only one pool per process and we cannot create a new one. The purpose of this limitation is to centralize all the asynchronous programming in the same pool, so that we do not have a third-party component that creates a parallel pool that we cannot manage and whose threads are degrading our performance.
Executing Functions on the Pool
The ThreadPool.QueueUserWorkItem method allows us to launch the execution of a function on the system thread pool. Its declaration is as follows:
The first parameter specifies the function that we want to execute on the pool. Its signature must match the delegateWaitCallback: