Book marketing for real programmers

ForkJoinWorkerThread.java :

"... If you are unfamiliar with them, you probably want to read Herlihy and Shavit's book "The Art of Multiprocessor programming" ..."

The book seems to be JCiP on steroids (would you expect less from something Doug Lea himself refers to? :) ) with 100+ pages on theory (pretty standard topics with nice illustrations, Java-friendly examples and quite detailed explanations) and 500 pages of everything you always wanted to ask Doug about (personally, I would not expect to understand most of the answers anyway, at least before reading this book).

It took me five cover-to-cover readings of JCiP (guess who is the slow one in my family? :) ) to get sufficiently comfortable with the subject. And I still occasionally re-read parts of it. AoMP addresses much harder questions although is probably less practical unless you are writing your own j.u.c or want to understand how the real one works.

Programming in Scala on Actors

I am through my first reading of the book and somehow it seems to be less helpful than I hoped it would be. I do not think it is bad or merely repackaged online documentation but I guess I expected more from a book written by the language author. Sublime wisdom or something :)

I believe most people are interested in Scala because it is an OOP-friendly tool for learning practical FP and Actors. So I was particularly irritated by chapters on GUI (who cares?) or minute details on integration with Java (better to be online instead of wasting precious book space). On the positive side, my feeling is that FP is covered in the book adequately.

What truly disappointed me was the chapter on Actors. To begin with, why only 1 chapter on this exciting topic? It is not like best Actor practices are better known than FP among us mere mortal with mostly C++/java background. And in my opinion that event simulator example is actually a good illustration of what not to do.

What I wanted to see was a clear comparison with traditional concurrent designs. So some universally-known concurrent system (e.g. a network /web server) would not require any explanation what it is and would immediately conjure up many examples from java world. Some code snippets would be probably even useful in real life. Not only modelling circuitry won't, it actually complicates the discussion with essentially unrelated clutter.

In addition, it would be really useful to learn how all this Actor machinery actually works. Is it some native magic or just a Scala library? How does it compare with j.u.c-based abstractions, at least performance-wide? Not a single word.

As a matter of fact, there is a much more elucidating paper named Actors that Unify Threads and Events. It really helps to read it before you delve into the dark internals of Actor.scala. It turns out that some interesting (and somewhat dirty? or truly ingenious? exceptions for control flow ..) design decisions were made to make it work.

As for the Actor-related code itself, I found it quite messy. Makes me wonder if the root cause was extreme performance fine-tuning or certain sloppiness. There is even a recent post where an independent Scala enthusiast comes up with a more orderly design of Actor.scala (and provides a few more hints on the official design). Hopefully Martin&Co will listen to him.

Interview-quality concurrent primitives

The exciting thing about interviews is that you hardly ever know how any of those will be conducted. Take concurrency as an example. It could be ignored, some people may ask you about plain vanilla wait/notify semantics, other about using primitives from the util.concurrent library, and then someone is likely to ask you about arguably more intellectual algorithmic side. In the latter case, a typical task is to implement something like a thread-safe bounded buffer or a read-write lock.

Well, I guess except for Messrs Lea and Goetz few would probably attempt to roll out a production quality CAS-based design having just a white board and half an hour. Which means that knowing simpler approaches makes sense at least in the context of interviews. Here where a simplistic book can help.

A half of it is dedicated to the RTSJ which seems to be more of a research toy. The other half provides a rather messy introduction to concurrency. I used to consider this book to be mostly useless (too thin theoretically, insufficiently util.concurrent-oriented for real-life development). But a few basic implementations of primitives such as semaphores and locks certainly can be used for interview purposes.

Distributed algorithms

Distributed algorithms are a fascinating but often overlooked area in which academic research meets real-life challenges. Well-known practical applications such as the JGroups framework tend to simplify (for performance reasons obviously, certain levels of consistency are prohibitively expensive in terms of network traffic or latency) the approaches suggested by academicians. On the other hand, virtually all such frameworks are firmly rooted in hard-core research (e.g. Bela Ban was influenced by research done in Cornell where Ken Birman seems to be the founding guru) and you won't go far on common sense and re-invented wheels.

In mainstream development layers upon layers of middleware protect programmers from the challenges of real engineering. Quite few are lucky enough to closely work with JGroups-level components and even fewer use NIO and MINA-like frameworks (BTW MINA's lead developer has just been hired by JBoss) to build infrastructure of their own. In my experience, even among CS graduates not everyone has a good grasp of this field. Once upon a time I struggled mightily to learn basics of reliable messaging and its internals. As a result, I have been keeping my eye on the literature but surprisingly enough for our age of grid computing there are only a few sources catering to practicing engineers. Most CS books tend to jump to the theoretical aspects and never come back to something more practical.

So far I can truly recommend only a couple of books paying attention to real code.

• Concurrent and distributed computing in Java - a very good introductory book covering both essential concurrency issues and distributed algorithms. The author did a very good job of actually explaining many concepts and their simple Java implementations (the code is probably not perfect but it makes so much easier to grasp the ideas) without overwhelming the reader with gratuitous mathematical rigor. The book consists of chapters of roughly equal size which makes it possible to read in manageable chunks. Each chapter refers to selected original articles and so explains the history of the field, again very succinctly.

• Introduction to Reliable Distributed Programming - this book is based on a real-world messaging framework and as such provides both an in-depth discussion of algorithms and their implementation. The only thing I can say is that I wish there were a comparable book on JGroups which is arguably the leading Java framework.If I live long enough to overgrow those two books I will probably read also Design and analysis of distributed algorithmsbut I am not entirely convinced it will happen before I retire :)

As a side-note, I found the JGroups source code rather messy. I frequently saw places where refactoring was long due. Consequently, I doubt that framework is a good educational instrument at least for low-level mechanics especially if you do not have a pretty good idea of what exactly to expect. My UML sketch of TCP/UDP stacks looks rather unwieldy but they say most highly useful pieces of software look puzzling inside.

TDD

In my opinion test-driven development and refactoring are the most important contributions of the Agile movement. Personally, I find the movement itself rather controversial and oriented on less complex systems. I believe that school of thought brings in a few valuable ideas from cognitive and industrial psychology. On the flip side, I am highly suspicious of their lack of engineering rigor.

Throughout my career I have seen different development organizations. Some took unit testing for granted (sadly, TDD was not explicitly encouraged but even sufficiently extensive unit tests go a long way). Others only briefly dabbled with JUnit. The latter invariably turned out to work on profoundly boring applications with little to brag about in the quality attributes department. Naturally, this experience taught me to screen potential employers by their attitude to unit testing and test coverage tools.

Most of the time the refactoring part of the equation was mostly the matter of taste of individual developers. It is easy to justify but certainly much more challenging to amend. I think refactoring is so intrinsically intertwined with artsy questions of style, readability and proper OOD that it is destined to be a way of thinking and not merely a cookbook. If you remember the founding book on the subject, virtually each and every refactoring there is accompanied with a counterpart reversing the action - you choose which one to apply in a particular situation.

In my experience things that are difficult to formalize are easier to learn by watching. Probably this is the only way and that's why it takes so long for a developer to mature despite a relatively narrow skill set required typically. Such learning tends to evolve through time and requires good reference materials to learn from.

I am reading a couple of books on the subject, namely Test Drivenand xUnit Test Patterns. Clearly, they are not comparable - the former is an also-ran, the latter is essentially the bible from a world-renowned series.

From Test Driven I would recommend an interesting chapter on testing concurrent code (thread-safety, blocking operations, starting and stopping threads, asynchronous execution). I hesitate to come up with another book addressing it.

Strangely enough, I found Test Patterns to be rather boring. Probably because of its 950 pages :) It reads more as a catalog of test idioms ("five ways to create a test fixture - transient, persistent, shared ..") and I was overwhelmed in my attempts to find and learn quickly a few new useful tricks. Surely enough the book is valuable, especially as a conceptual guide, but do not expect much fun.