Final Study Guide
Parallel Computing I - Topics 20032

1. General
   • Why parallel computing now? (Reference Lester/Wilson/Barry and others)
   • Hardware Issues
     • Flynn's Taxonomy
     • Global vs Distributed Memory
     • Communication/memory networks
     • Granularity (# of processors)
   • Software Issues
     • Mapping Algorithms to Architecture
     • Languages
     • Compilers (Automatic)
     • Operating Systems
     • General Purpose
     • New Methods (algorithm design)
     • Correctness/debugging
     • Granularity (problem decomposition)
   • Performance Issues
     • Granularity
     • Load Balancing
     • Parallel Constructs vs Architecture
     • Computation vs Communication

2. Parallel Languages
   • Parallaxis
   • Linda
   • MPI
   • FORTRAN (automatic vs subroutine calls vs compiler directives)
   • OpenMP
   • Issues
     • Implicit Parallelism
     • Explicit Parallelism
     • Parallel Languages
     • Portability vs Efficiency

3. Metrics (references Quinn/Lester/Wilson)
   • Speedup
   • Efficiency
   • Amdahl's Law
   • Cost

4. Network Topologies
   • Topologies
   • Why hypercube was so popular
   • Diameter
   • Grey Codes
   • Dynamically Configurable Networks
     • Crossbar
     • Multistage
   • How relates to issues with memory

5. Parallel Algorithms/Applications
   • Designing Parallel Algorithms (Reference Quinn)
     • Summation/Max/Min
   • From SIMD to MIMD
     • Odd-Even Transposition Sort
     • Parallel Prefix (Didn't cover)
   • 1-D Wave Equation (Reference - Fox etal.)
     • Communication Template
     • Communication Overhead
     • Problem Decomposition
     • Problem Dimensionality and Performance Metrics
   • Matrix Multiplication (Reference - Quinn)
   • Parallelization of iterative techniques (Reference - Bertsikas, etc.)
   • Parallel Sorts
   • Parallel graphics
   • FFT's (didn't cover)

6. Heterogeneous and Cluster Computing
7. Future Parallel Computers?
   • Quantum Computing
   • DNA Computing
   • Optical Computing

---

Nan C. Schaller
Rochester Institute of Technology
Computer Science Department
102 Lomb Memorial Dr.
Rochester, NY 14623-5608
telephone: +1.585.475.2139
fax: +1.585.475.7100
e-mail: ncs@cs.rit.edu
November 24, 2003