Spring2025: COSC 4397 (Selected Topics in Computer Science)

Parallel Computations of GPU

Lecture Tu/Th 1pm-2:30pm: Location SEC104
Instructor: Panruo Wu (pwu7@uh.edu)

Reference books:

• Programming Massively Parallel Processors, A Hands-on Approach, 4th Ed
• Computer Systems: A Programmer's Perspective, 3/E

Reference online resources

• Doc: CUDA C++ Programming Guide
• Video course: CUDA Programming Course – High-Performance Computing with GPUs
• Video: Andrej Karpathy: LLM.c, LLM training in C/CUDA
• Book: CUDA by exmaples

Intro

As modern hardware increasingly depends on explicit parallelism to deliver performance, modern computer scientists and practitioners need to acquire programming and parallelization skills to develop efficient and scalable systems and applications. General purpose GPUs (GPGPUs) are the most accessible, powerful, and widely used platforms in several scenarios where performance is critical, including large neural network training/inference, advanced cryptography, and scientific/engineering simulation.

This course teaches fundamentals on GPGPU architecture, parallel computation principles, and the methods of achieving very high performance on GPU via programming and performance engineering. The main vehicle of teaching programming on GPU is CUDA on NVIDIA GPUs. The goal of this course is to be able to identify problems that can be parallelized effectively and realize parallel computation on massively parallel systems such as GPUs, and techniques to achieve very high performance on GPUs.

This will be a pretty hands-on course--so get ready to get hands dirty! A NVIDIA GPU machine will be provided for you to use for this course. The main programming tools are CUDA on NVIDIA GPUs.

Schedule

This schedule is tentative! It will be adjusted frequently. Last modified: Jan 22, 2025