In this course, you will be introduced to computing systems, software, algorithms, and methods used to solve large-scale problems in science and engineering. We will discuss algorithms as well as tools for high-performance computing.

The principal objective of this course is to introduce you to the use of high-performance computing systems in science and engineering. This course is structured to train students to reason about the design and implementation of efficient parallel programs. The focus areas of the class will be on the modeling, implementation and evaluation of distributed, message-passing interface (MPI) based programs, shared-memory thread-based OpenMP programs, and hybrid (MPI/OpenMP) programs.

Here is a tentative list of topics we will cover in this class:

  • Parallel programming models
  • Shared memory (Work/Depth, PRAM)
  • Sorting, searching, selecting, merging
  • Dense and sparse linear algebra
  • Discrete algorithms
  • Trees, graphs (coloring, partitioning, traversal)
  • Numerical algorithms
  • Butterfly algorithms (FFT)
  • Domain decomposition
  • Multigrid
  • N-body Algorithms (Fast Multipole Methods, Tree codes, nearest neighbors)

Programming APIs

  • Shared-memory (OpenMP and PThreads)
  • Distributed memory (message passing, MPI)
  • GPUs (CUDA/OpenCL)