Dispelling the myths of parallel computing

Some myths and misconceptions of parallel computing are discussed. The first myth to consider is the idea that scalability, the ability of an application to utilize large numbers of processors to reduce run time linearly, is equivalent to high performance. Scalability is sometimes measured as processor utilization, or by the number of Floating Point Operation per Second (FLOPS) achieved. It should be noted that while parallelism might recover some of the run time loss due to algorithmic inefficiency, it does not recover power consumption losses. A second myth to consider is the notion that high performance can be achieved by focusing on sections of parallel code, and that other sections of an application can be dismissed. The final myth we consider is the notion that challenges to scalability can be achieved by simply working harder, having programmers be more innovative, developing new architecture designs that minimize communication delays, and so on.