Eli Bendersky's website The authors - all senior software engineers at Intel (an important factor I'll come back to later), attempt to come up with a "pattern language" for parallel programming, similar to the gang of four's Design Patterns of OOP. This is a daring attempt, and while the book certainly has some good things going for it, I think that the end result is fairly mediocre.
First, what I liked about the book:
- The introductory part (first two chapters) are well written and paced just right to serve as a good introduction to the topic - not too wordy, not too terse.
- The list of patterns is comprehensive and definitely provides a good starting point for a common language programmers can use to talk about parallel programming. Folks experienced with parallel programming throw around terms like "map", "scatter-gather" and "stencil" all the time; if you want to know what they are talking about, this book has a good coverage.
- At least some of the examples are interesting and insightful. Especially the first example (Seismic simulation) is enlightening in its use of space-time tiling. This is actually the kind of topic I really wish the book spent more time on.
- The formatting is good: diagrams are well executed and instructive, and the many code samples use consistent C++ style and are comprehensive.
Now, what I didn't like:
- First and foremost, and this is a criticism that permeates throughout this review: this book is thinly veiled marketing material for Intel. Except for one OpenCL example, the authors just use OpenMP, TBB, ArBB and Cilk Plus to demonstrate the patterns - all Intel-specific techonologies that are only optimized for Intel CPUs. If you care about more HW than Intel CPUs, or are using a different programming model/library, you're out of luck.
- The book takes a very narrow view of parallel hardware, which is IMHO unforgivable for something written in 2012. For many parallel workloads these days, GPUs offer a much better performance/cost alternative to CPUs. Naturally, you won't have Intel engineers admit it in their book. GPUs are mentioned only very briefly in the introduction, and then only to shame them being less flexible than CPUs. This is despite the fact that many of the patterns presented in the book are actually perfectly good for GPUs. The authors do mention Intel's MIC, of course. But MIC, 4 years after the book has been written, still very much looks like a fringe technology which is inferior to Nvidia's server-class GPUs for number crunching.
- The book also takes a very narrow view of software. Some of the Intel technologies it presents are already defunct - like ArBB. Others, like Cilk Plus are so esoteric and rarely used that only Intel still doesn't realize it. TBB is probably the most reasonable of the technologies presented, since it's an open-source library. If the book used plain threads and then showed what TBB brings to the table, it would be significantly more useful, IMHO.
- The actual "meat" part of the book is extremely short. After the introduction, less than 200 pages are spent listing the patterns, and maybe half of that is dedicated to discussing the idiosyncracies of the specific Intel technologies the authors use to implement them.
- Distributed computing is completely neglected - only shared-memory models are discussed. If you want to break a task up into multiple machines that don't share memory, this book won't help you much (except a brief mention of map-reduce towards the very end).
Overall, I won't say this is a bad book - there's certainly useful information within it and it's well written. But it's also far from being a great book. Maybe if all your parallelism needs are confined to a single multi-core Intel CPU and you're happy to use one of the Intel technologies it covers, the book can be great. Another audience which can take more from the book is relative beginners who had only basic exposure to parallel programming - then the patterns are truly useful to know about.
I'll be happy to hear suggestions about great books on parallel programming. Being such an important topic these days, it's surprising to me that I can't find a single book that's universally recommended.