Parallel Computing

Parallel computing is about taking a problem that's too slow or too big for one core and figuring out how to split the work across many — which sounds simple until you hit the real constraints of communication cost, load imbalance, and overheads that quietly erase your speedup. Across four projects you'll write actual parallel code in the three paradigms that matter in practice (MPI for distributed memory, OpenMP/threads for shared memory, CUDA for GPUs), and learn to reason about scalability formally through metrics like efficiency, isoefficiency, and cost-optimality rather than just timing your runs. Sitting on top of CS 223/CS 342, it's the course that turns "I know how computers work" into "I can make a cluster or a GPU actually deliver its peak," which is the baseline skill for HPC, ML systems, and anything compute-bound.