From-Scratch Build · HPC Curriculum
The second book I'm writing from scratch to learn high-performance computing — where the foundations get pointed at real problems: fluid dynamics, climate, finance, health, deep learning at scale, and the frontier of quantum and neuromorphic machines. My own chapters, examples and design.
What it is
This is the companion to my foundations volume. Where the first book teaches the machinery — architecture, parallelism, OpenMP and MPI — this one asks the harder question: what is all that power actually for? I wrote it to learn how supercomputing turns into real science and engineering.
The chapters split into two halves. The first sharpens the tools — advanced MPI and OpenMP, accelerators like GPUs and FPGAs, hybrid programming, and serious code optimisation. The second points those tools at domains: simulating airflow, modelling the climate, pricing in finance, analysing the brain, training neural networks across many GPUs, and finally the emerging machines — quantum and neuromorphic — that may be the next frontier.
The core idea I wanted to learn: the hardest part of applied HPC isn't the physics or the model — it's mapping a real problem onto thousands of processors so it actually scales. Every domain chapter is really a lesson in decomposing a problem until the machine can chew through it.
The toolkit
Each was new to me, and each unlocks a different class of problem. Here is what each one actually does.
Non-blocking communication, collectives and topologies — the techniques that keep message passing efficient when you go from a handful of processes to thousands.
Specialised processors for massive parallelism. GPUs for data-parallel maths; FPGAs for custom, reconfigurable hardware pipelines.
Real clusters mix all three layers at once. Hybrid patterns let one program use shared memory, message passing and accelerators together.
Large scientific runs are pipelines, not single jobs. Orchestration tools chain steps, manage data and make runs reproducible.
Training one model across many GPUs and nodes by splitting the data or the model itself — the same HPC ideas applied to neural networks.
The emerging paradigms: qubits exploiting superposition, and brain-inspired chips that compute with spikes instead of clocks.
The chapters
The volume climbs from advanced technique, through applied domains, to the genuine frontier. The final stage is where I'm still drafting — fittingly, it's the part nobody has fully figured out either.
Deeper OpenMP and MPI, plus accelerators and FPGAs — pushing the foundations to their limits.
Tuning code for real architectures and choosing data layouts that the hardware can actually fly through.
Turning one-off runs into automated, reproducible pipelines across a cluster.
Computational fluid dynamics and climate / terrestrial modelling — classic HPC workloads written out as case studies.
Finance, health and neuroscience, plus deep learning at scale: where HPC meets large data and AI.
Digital twins, neuromorphic computing and quantum computing — the chapters I'm still actively writing.
How I work on it
Each applied chapter follows the same loop — pick a real problem, find the parallel structure hiding in it, and only then write:
The chapters live as notebooks so the science, the code and the results stay together — read the problem, then watch a scaled-down version actually run with mpirun or on a GPU.
Reflection