Mega Code Archive

Understanding Threads

Title: Understanding Threads Question: Understanding Threads, Threading Models and MultiThreading Answer: Thread Overview ================== What is a thread? The term thread can be said as a short term for thread of control, and a thread of control is, a section of code executed independently of other threads of control within a single program. Thread of Control ================== Thread of control sounds like a complicated technical term, but it's really a simple concept: it is the path taken by a program during execution. This determines what code will be executed: does the if block get executed, or does the else block? How many times does the while loop execute? If we were executing tasks from a "to do" list, much as a computer executes an application, what steps we perform and the order in which we perform them is our path of execution, the result of our thread of control. Having multiple threads of control is like executing tasks from two lists. We are still doing the tasks on each "to do" list in the correct order, but when we get bored with the tasks on one of the lists, we switch lists with the intention of returning at some future time to the first list at the exact point where we left off. Overview of Multitasking ========================= We're all familiar with the use of multitasking operating systems to run multiple programs simultaneously. Each of these programs has at least one thread within it, so at some level, we're already comfortable with the notion of a thread in a single process. Now consider what happens when you sit at your computer and start two single-threaded programs: a text editor, say, and a file manager. You now have two processes running on your computer; each process has a single thread with the properties just outlined. Each process does not necessarily know about the other process, although, depending on the operating system running on your computer, there are several ways in which the processes can send each other various messages. A common behavior is that you can drag a file icon from the file manager into the text editor in order to edit the file. Each process thus runs independently of the other, although they can cooperate if they so choose. The typical multitasking environment is shown in Figure 1-1. Figure 1-1. Processes in a multitasking environment [Find Attached zip file] From the point of view of the person using the computer, these processes often appear to execute simultaneously, although many variables can affect that appearance. These variables depend on the operating system: for example, a given operating system may not support multitasking at all, so that no two programs appear to execute simultaneously. Or the user may have decided that a particular process is more important than other processes and hence should always run, shutting out the other processes from running and again affecting the appearance of simultaneity. Finally, the data contained within these two processes is, by default, separated: each has its own stack for local variables, and each has its own data area for objects and other data elements. Under many operating systems, the programmer can make arrangements so that the data objects reside in memory that can be shared between the processes, allowing both processes to access them. Why Threads? ============ Historically, threading was first exploited to make certain programs easier to write: if a program can be split into separate tasks, it's often easier to program the algorithm as separate tasks or threads. Programs that fall into this category are typically specialized and deal with multiple independent tasks. The relative rareness of these types of programs makes threading in this category a specialized skill. Often, these programs were written as separate processes using operating-system-dependent communication tools such as signals and shared memory spaces to communicate between processes. This approach increased system complexity. The popularity of threading increased when graphical interfaces became the standard for desktop computers because the threading system allowed the user to perceive better program performance. The introduction of threads into these platforms didn't make the programs any faster, but it did create an illusion of faster performance for the user, who now had a dedicated thread to service input or display output. Recently, there's been a flurry of activity regarding a new use of threaded programs: to exploit the growing number of computers that have multiple processors. Programs that require a lot of CPU processing are natural candidates for this category, since a calculation that requires one hour on a single-processor machine could (at least theoretically) run in half an hour on a two-processor machine, or 15 minutes on a four-processor machine. All that is required is that the program be written to use multiple threads to perform the calculation. While computers with multiple processors have been around for a long time, we're now seeing these machines become cheap enough to be very widely available. The advent of less expensive machines with multiple processors, and of operating systems that provide programmers with thread libraries to exploit those processors, has made threaded programming a hot topic, as developers move to extract every benefit from these new machines. Until Java, much of the interest in threading centered around using threads to take advantage of multiple processors on a single machine. Independent Tasks ================== The complete answer to the question "Why threads?" really lies in this category. As programmers, we're trained to think linearly and often fail to see simultaneous paths that our program might take. But there's no reason why processes that we've conventionally thought of in a single-threaded fashion need necessarily remain so: when the Save button in a word processor is pressed, we typically have to wait a few seconds until we can continue. Worse yet, the word processor may periodically perform an autosave, which invariably interrupts the flow of typing and disrupts the thought process. In a threaded word processor, the save operation would be in a separate thread so that it didn't interfere with the workflow. As you become accustomed to writing programs with multiple threads, you'll discover many circumstances in which adding a separate thread will make your algorithms more elegant and your programs better to use. Threading Background ===================== There are three basic models of threading in the Win32 environment: single, apartment, and free. Single Threading ================ Some of the first applications that you wrote were probably single threaded, with the only thread corresponding to your application's process. A process can be defined as the instance of an application that owns that application's memory space. Most Windows applications are single threaded with a single thread doing all of the work. Apartment Threading =================== Apartment threading is the next level of complexity in threading models. Code marked for apartment threading can execute on its own thread, confined to its own apartment. A thread can be defined as an entity owned by a process that is scheduled for processing time. In the apartment-threading model, all threads operate within the confines of their own subsections of the main application's memory. This model allows multiple instances of your code to execute simultaneously but independently. For example, before .NET, Visual Basic was limited to creating apartment-threaded components and applications. Free Threading =============== Free threading is the most complex threading model. In the free-threading model, multiple threads can call into the same methods and components at the same time. Unlike apartment threading, they are not confined to separate memory spaces. One instance where you might use a free-threaded object is when an application must do a large number of very similar yet independent mathematical calculations. In this case, you would spawn multiple threads to perform the calculations using the same code instance. Working with Threading Models ============================= One way to conceptualize the threading models is to consider the task of moving from one house to another. If you take a single-threaded approach, you do all the work yourself, from packing to carrying boxes to unpacking. If you operate in an apartment-threaded model, you invite some of your closest friends to help you. Each friend works in a separate room and cannot help the people working in other rooms. They are in charge of their own space and moving supplies. If you go for a free-threaded approach, you would invite those same friends over to help, but all of them work in any of the rooms at any time, sharing packing supplies. In this analogy, your house is the process in which all your threads operate, each of your friends is an instance of your code, and the moving supplies are your application's resources and variables. This example illustrates some advantages and disadvantages of the different models. Apartment threading is faster than single threading because multiple instances of your component are doing the work. Free threading is faster and more efficient than apartment threading in some situations because everything is happening at once and all the supplies are shared. This can lead to problems, however, if multiple threads are changing shared resources. Imagine that one person started to use a box and packed it with items from your kitchen, and then another friend came along and used that same box to pack things from your bedroom. The first friend labeled the box 'Kitchen,' and then your second friend overwrites that label with 'Bedroom'. When you unpack, you will end up with kitchen supplies in your bedroom. Why Multithreading? ================== To understand the power of multithreading you need to know something about how the Windows operating system works under the hood. Windows is a preemptive multitasking operating system. The system CPU can do only one thing at a time, but to give the illusion that multiple processes are running simultaneously the operating system splits the CPU time between the various running processes. The term preemptive means that the operating system determines when each task executes and for how long. Preemptive multitasking differs from cooperative multitasking where each task decides how long it will execute. Preemptive multitasking prevents one task from taking up all the processor's time. For example, you might think you can edit a word processing document, print a spreadsheet, and download an MP3 file from the Web all at the same time, but in actuality, the operating system allocates small "slices" of CPU time to each of these processes. Because the CPU is very fast, humans are very slow, and the time slices can be extremely small, all three processes appear to run simultaneously. On a multi-processor system things are a bit more complicated but the basic idea is the samethe operating system divides the time of the CPUs among the processes that need it. Each process has a priority that determines how the operating system allocates CPU time to it relative to other processes. Schemes for allocating process priorities differ among OS's, but Windows 2000 has four priority levels. In descending order, these priorities are: Real time: The highest priority. These processes preempt all processes with lower priority. Real time priority is reserved for processes that cannot suffer even minor interruptions, such as streaming video and games with complex graphics. High priority: Used for time-critical processes that must execute immediately (or almost immediately) for proper operating system functionality. The Windows Task List is an example of a high-priority process. It is assigned high priority because it must display immediately when requested by the user regardless of anything else the operating system is doing. Only real-time priority threads can interrupt a high priority thread. Normal priority: Used for processes that have no special CPU scheduling needs. Word processing and background printing are examples of normal priority processes. Idle priority: Used for processes that run only when the system is otherwise idle. A screen saver is a good example of this priority level. These thread priorities are those defined for the Windows 2000 operating system. Here's where the concept of threads becomes important. A thread is the unit, or entity, to which the operating system assigns CPU time. Normally a program or process is associated with a single thread. The process must accomplish everything it needs to dodrawing the user interface, responding to user input, writing files, calculating results, everythingduring the CPU time that Windows allocates to its single thread. For most applications this works just fine. Windows also supports multi-threaded processes. As the name suggests, a multi-threaded process is one in which the program divides its tasks among two or more separate threads. Because Window allocates processor time to threads and not to processes, a multi-threaded process gets more than the usual share of CPU time. So is the purpose of multi-threading simply to speed up a program? No, although it can have that effect, particularly on a multiprocessor system. The most important use of multi-threading has to do with the program's responsiveness to users. There are few things more frustrating to users than a program that does not respond immediately to mouse and keyboard input. Yet, when a single-threaded program has lengthy calculations or I/O activities going on at the same time this is exactly what is going to happen. The program's one thread must handle user interaction and the calculations, which often causes the user interaction to become sluggish. Therefore, when your program must respond to user input at (perceptually) the same time that it is performing one or more other tasks, you should consider multi-threading. By assigning calculations or I/O to one thread, and user interaction to another thread, you can create a program that responds to user actions efficiently while also performing the required data processing. The concept of threads is central to the inner workings of most operating systems, including Windows, but relatively few programmers even know what they arelet alone how to take advantage of them. Understanding threads, using modern operating system's multi-threading capabilities properly, is a fundamental step toward creating fast, responsive applications. There's Always a Downside ========================== Using multiple threads is not all roses, however. As with almost everything else in life, a penalty accompanies the advantages. There are several factors you should consider. First, keeping track of and switching between threads consumes memory resources and CPU time. Each time the CPU switches to another thread, the state of the current thread must be saved (so it can be resumed again later) and the saved state of the new thread must be restored. With too many threads any responsiveness advantages you hoped to gain may be partially nullified by the extra load placed on the system. Second, programming with multiple threads can be complex. Creating a single extra thread to handle some background calculations is fairly straightforward, but implementing many threads is a demanding task and can be the source of many hard-to-find bugs. My approach to these potential problems is to use multithreading only when it provides a clear advantage, and then to use a few threads as possible. Third is the question of shared resources. Because they're running in the same process, the threads of a multi-threaded program have access to that process's resources, including global, static, and instance fields. Also, threads may need to share other resources such as communications ports and file handles. You must synchronize the threads in most multi-threaded applications to prevent conflicts when accessing resources, such as deadlocks (when two threads stop as each waits for the other to terminate). For example, suppose that Thread A is responsible for obtaining data over a network, and Thread B is responsible for performing calculations with that data. It might seem like a good idea to require Thread A to wait for Thread B to complete (so the data is not updated in the middle of a calculation), and also to require Thread B to wait for Thread A to complete (so the latest data is used in the calculations). If coded improperly the result will be two threads that never execute. Issues with Multithreaded Code =============================== Threading can dramatically improve both the performance and scalability of some applications. With this power, you must be aware of some dangers of threading. A few situations where using threads can be detrimental to your application do exist. Threads may down operation, cause unexpected results, or even cause your application to stop responding. If you have multiple threads, make sure that they are not waiting for each other to reach a certain point or to finish. If not done correctly, this can lead to a deadlock state where neither thread will ever finish because each is waiting for the other. If multiple threads will require access to a resource that cannot be shared easily, such as a floppy disk drive, serial port, or an infrared port, you probably want to avoid using threads or use one of the more advanced threading tools such as synclocks or mutexes to manage concurrency. If two threads attempt to access one of these resources at the same time, then one will fail to acquire the resources or cause data corruption. Another common problem with threading is a race condition. If one thread is writing data to a file while a second thread is reading from that file, then you do not know which thread will finish first. This is referred to as a race condition because the two threads are racing to the end of the file. If the reading thread were to get ahead of the writing thread, then unknown results would be returned. When using threads, you should also consider whether all the threads would be able to work completely independent of each other. If they do need to pass data back and forth, you want to be careful that this data is relatively simple. When you pass complex objects, you start to incur heavy marshaling costs to move those objects back and forth. This will generate overhead for the operating system to manage as well as reduce general performance. Another concern is the transition cost of handing off your code to another developer. Note that the next developer who maintains your code must understand threading to work with it. While this is not a reason to avoid threads, it is an excellent reason to supply adequate code comments. While none of these issues should discourage you from threading by itself, they are all issues you should keep in mind when you are designing your application and deciding whether to use threads or not. If you have decided to use threads but are running into some of the problems mentioned above, check out synclocks or mutexes to see if they will solve your problem or lead you to another solution.