Programming in Scala on Actors

I am through my first reading of the book and somehow it seems to be less helpful than I hoped it would be. I do not think it is bad or merely repackaged online documentation but I guess I expected more from a book written by the language author. Sublime wisdom or something :)

I believe most people are interested in Scala because it is an OOP-friendly tool for learning practical FP and Actors. So I was particularly irritated by chapters on GUI (who cares?) or minute details on integration with Java (better to be online instead of wasting precious book space). On the positive side, my feeling is that FP is covered in the book adequately.

What truly disappointed me was the chapter on Actors. To begin with, why only 1 chapter on this exciting topic? It is not like best Actor practices are better known than FP among us mere mortal with mostly C++/java background. And in my opinion that event simulator example is actually a good illustration of what not to do.

What I wanted to see was a clear comparison with traditional concurrent designs. So some universally-known concurrent system (e.g. a network /web server) would not require any explanation what it is and would immediately conjure up many examples from java world. Some code snippets would be probably even useful in real life. Not only modelling circuitry won't, it actually complicates the discussion with essentially unrelated clutter.

In addition, it would be really useful to learn how all this Actor machinery actually works. Is it some native magic or just a Scala library? How does it compare with j.u.c-based abstractions, at least performance-wide? Not a single word.

As a matter of fact, there is a much more elucidating paper named Actors that Unify Threads and Events. It really helps to read it before you delve into the dark internals of Actor.scala. It turns out that some interesting (and somewhat dirty? or truly ingenious? exceptions for control flow ..) design decisions were made to make it work.

As for the Actor-related code itself, I found it quite messy. Makes me wonder if the root cause was extreme performance fine-tuning or certain sloppiness. There is even a recent post where an independent Scala enthusiast comes up with a more orderly design of Actor.scala (and provides a few more hints on the official design). Hopefully Martin&Co will listen to him.

Is Java relevant?

My previous post made me think about my attitude towards Java. Even if it happens to be behind its pinnacle, is Java any good beyond paying my bills for years? If we know what Java gave this world probably we will be better equipped to decide when to move to a new language and how to judge it.

History teaches us that non-technical aspects matter more than what is habitually referred to as technical excellence. Which is actually difficult to define. I have a growing suspicion that what matters a lot is something I would call, for a lack of standard term, "age and experience demographics". The idea is that certain technologies are perceived to be cool by people entering the field. Those technologies in a sense create an imprint on that particular generation.

As an example, in the early nineties Assembly was for real men and C++ for real programmers. At least how I felt it back then under certain circumstances (no influence from CS-savvy people, and extremely limited literature in the absence of the Internet, most people using and targeting MSFT platform). I know that years before I discovered computers people had been writing in HEX code and considered Assembly to be unmanly. It very well may be that nowadays something like scripting/dynamic languages are all the rage among the young.

There must be also a heavy influence of experience resulting in some kind of "shifting interest curve". You need to be through a complete life cycle of a major language such as Java to be able to see that not everyting is new under the sun. As an example of a self-made developer, I have progressed through Pascal, C++, OOP, OOA&D, RUP, UML, Java, algoritms and data structures 101, development methodologies, software architecture, really understanding OOA&D, really understanding Java concurency, TDD (disclaimer: for simplicity sake the sequence was artificially serialized and stripped of many things).

Right now I value good analysis and design (assuming OOA&D; I am not aware of FP/dynamic language alternatives to it; and yes, I appreciate UML) and best software engineering practices more than a particular language. I like reading about algorithms much more than I did ten years ago. I am dabbling with FP in its safely diluted incarnation (Scala) and planning to look again at Erlang later next year. Who knows, probably in my next ten years I will be able to get interested and even like highbrow stuff such as LISP. The point is that in addition to normal technological progress each individual has another learning curve which includes things not affected directly by his current job.

And last but not least, not everyone majored in CS. As a matter of fact, I believe I have worked with more people from Engineering/Physics backgrounds than properly educated CS graduates. Naturally, the former were not indoctrinated in seemingly non-practical subjects such as LISP, FP or even the details of compiler design. Some of those are probably later picked in spare time but I would still expect a gap between originally CS and non-CS folks.

So, what do I like about Java? I happen to maintain an Apache module written in C and every time I fix a bug there I appreciate our warm and fuzzy Java world more. This juxtaposition elucidates good things about Java although by no means it is even remotely scientific or exhaustive. Disclamer: borders between Java the language, the JVM and Java ecosystem at large are blurred to reflect real life.

• Java has a standard API (or a de facto standard API) for nearly everything. And innumerate open-source implementations of those.
  
• Java has multiple open-source components available for anything from reliable messaging and IoC to web UI and Map/Reduce on serious scale.
• We have a set of widespread mature technologies covering most important mundane areas (e.g. Maven/Spring/JUnit)
• Java has a memory model and an amazing util.concurrent library. It was Doug Lea and not James Gosling who was my youth hero :) And Brian Goetz wrote really nice javadocs :)
• Java has a decent Collections libraryJava dominates the server side. Jobs are abundant in interesting domains from Finance to Telecommunications
• Java promotes OOP and interface-based design. Not really design by contract though :(
• Because Java is a dumbed-down language, it is easy to read (well, multi-screeen methods are still found out there but I guess you catch my drift). Though I have always missed overloaded operators and proper generics (enabling the STL library in C++).
• Nowadays there are enough high-quality Java books
• Without IntelliJ IDEA I would go out, shoot myself in the foot and die :)
• Java does not look weird (presumably in comparison with the C branch of evolution)

Most of us would be able to continue this list for a long time. When I look at it the point seems to be indeed that it is not about Java the language. Libraries and the JVM sneak into the picture all the time. As a creature of habit I like Java because I am used to it but if a JVM language more sophisticated than Java comes along I would be all for it.

Harking back to Scala, the language looks smart to me (yes, I am enamored with C++ like complexity and things like covariant/contravariant type annotations attract me rather than scare away). I detest languages that restrict me (Java more than C++, Erlang more than Scala). I cannot imagine living with a language where I would not be able to do OOP even if I like the idea of trying out at least elements of FP. And Martin reminds meBjarne more than others do :)

Is Java obsolete?

This topic seems to be nearly obligatory nowadays in Java circles. Is Java a mature language about to enter its years of slow but inexorable decline? Can it be resuscitated? Should it be? Does anybody care? :)

Clearly, future will tell and there are more experienced people out there who can better articulate their predictions. As most people in my generation I moved to Java from C++ and from the very beginning I knew how many sophisticated features were missing in Java. Back then I was still learning the ropes so it is safe to say that Java is the first major language I have seen evolving from its early years. When I started using C++ it was already king of the castle . Although reading about its history was amusing I was too young to truly appreciate the concept of language life cycle.

Harking back to Java, I am more fascinated by historicity and sociological/psychological roots of programming language preferences and fashions. There are usually a few competing schools of thoughts in most technical areas. Thus, even if I tend to align with some of them I do not believe there is the only right answer. And I hear that quite often technical prowess does not always decide the fate of a technology. The longer I live the more I tend to see heavy influence of wetware on seemingly semi-scientific matter.

I remember talking to a colleague about it a couple of years ago. Under influence of the first(?) book on Java obsolescence I told him that we were not likely to write in Java in, say, ten years. I recollected the year 2000 when I myself was surprised why anybody cared about that obscure thing called Java. I was even more stunned when MSFT embarked upon cloning Java under amusical brand. Why anybody would want anything else if they already had C++?

Fast forward 8 years and more than once I have seen blogs saying that in a few years Java will be actively used only for "low-level, system development". The notion of "low-level" has changed a tad since the time I was young :) And to my personal taste, I'd rather stay in Java than do whatever will be classified as "high-level".

Lacking a CS degree I hardly heard about functional programming or LISP in my youth. And certainly never encountered such things in my professional life. When the recent FP fad started spreading through blogoshere I was mostly taken aback because I did not make much sense to me. And I was pretty irritated with the idea of learning a new language a year. Frankly, I still do not count SQL or Ant scrips as languages (unless implying DSLs) and I do not think a year of even full-time development is enough to learn a serious language (as my experience with C++ and Java tells me).

Actually what is meant by that suggestion is that there is apparently no general metamodel for features found in different languages. So there is no way to learn about a useful concept or idiom without actually spending a lot of time on learning to read languages one is unlikely to use in real life. Which sounds like wasting a lot of time to me. Actually, I prefer frameworks to languages. As an example, deep inside the Erlang runtime all those marvelous message queues and clustering features are actually implemented. I'd rather read a book on algorithms used in that implementation than learn Erlang simply to familiarize myself with those concepts. Alas, I have never seen any Erlang runtime design documentation.

I am also yet to find a paper clearly explaning how OOP and FP could be two sides of the same coin. As they intuitively seem to be (e.g. think about Erlang actors or modules as classes .. yes, strictly speaking it's far-fetched at best). Judging from the dearth of literature on FP design or UML analogs (or at least profiles) for FP people somehow manage to survive with methodologies originated in OOA&D. Which implies certain affinity.

Anyway, I am reading the first book on Scala. It's fun, it's not weird the way Erlang or LISP are, it definitely feels as Java 2.0 and it makes all kinds of Java deficiencies clearly visible. A few things look rather unnatural or necessary only for particular corner cases at first glance. But in general the language is definitely more sophisticated than Java. It also makes me believe that there is no reason to introduce major language features to Java anymore. It would feel as a different language and we would still have tons of legacy code which often does not use even generics. I did not think so even a year ago but I guess now I get what people were talking about for the last couple of years :)

There is no reason to cry about throwing closures out of Java7. Not everyone migrated even to Java6, not to mention those cursed with J2EE stuff chained to jdk1.4. It would be better to officially introduce, say, Scala as the next main JVM language and let Java peacefully lose its mind share. My guess is that we have learned enough from Java to not only wish for but also have a good idea of how a major overhaul of the JVM world should look like.

ctime: change or creation?

Trying to understand how Apache runtime manages to deal with three different timestamps I stumbled upon a fascinating discussion of the same question in the context of underlying Unix file system. It turns out to be even more confusing than I thought.

Networking frameworks revisited

Talking to a former colleague the other day I learned about recent changes in the world of NIO frameworks. The guy who led the MINA project joined JBoss and designed a new framework known as Netty. Previously, JBoss Remoting 3 was supposed to be based on MINA2.

Now the original MINA and Netty designer is on Remoting team as well so I feel I cannot make sense of all this proliferation anymore (even not taking into account Grizzly). It remains to be seen if some grand unification/survival of the fittest will ensue.

While looking at JBoss documentation I also found that they have a new NIO-related project. It makes me wonder how dead Java is if a single company can produce so much new stuff in a pretty obscure niche.

FP and LISP for the rest of us

As a developer with a traditional C++/Java background and a non-CS major I find the recent surge in popularity of functional programming pretty confusing. It turns out that there is a whole parallel universe of FP where most hard-won imperative experience is of little help in understanding new concepts. And those are frequently hidden behind bizarre syntax.

I have stumbled upon a couple of very well written articles on FP and LISP . The author starts from something as trivial as Java and Ant and ends up explaining, say, the power of LISP. Very approachable with quite a few insights.

Date and time in Java

Today I was fixing a bug which revealed a problem in how we incremented dates (yes, by adding 86400000). Naturally, it led to corrupted data after recent switch back to Pacific Standard Time from Pacific Daylight Time. I had to go review a very good overview of time-related Java issues and play with a date calculator.

Frankly, it's still rather counter-intuitive to me how much complexity is behind seemingly innocent long timestamps. What could be simpler than milliseconds passed since 1970? ;)

Event Stream Processing

I have just stumbled upon this new approach to event-based design. A wikipedia example says it all in 7 lines. The idea is to use an SQL-like language to create filters (capable of event correlation, typically limited to a certain period) and run a continuous stream of events through them. There is even an open source implementation. At first glance most currently available products are used in algorithmic trading (whatever was left of it after the crisis :) ).

To me this concept seems to be a kind of rule engine and so potentially there are multiple domains where it might be applied. On the other hand it makes me wonder whether it is indeed a re-packaged rule engine and in such a case what is so different about it.

Performance is the name of the game because ESP is positioned as a soft real-time technology. It must be pretty interesting to read how they manage to achieve their declared ability to process a huge number of events per second on commodity hardware.

Bay Area Scala Enthusiasts

There are a few user group-like communities I am aware of

Bay Area Scala Enthusiasts, Silicon Valley Java Users Group,Bay Area Functional Programmers Unfortunately they frequently meet somewhere on the peninsula which is a decisive no-go for people working in SF and commuting on BART.

Luckily, the latest Bay Area Scala Enthusiasts meeting was held at Twitter HQ in downtown SF yesterday. Naturally, I could not resist the temptation of walking 0.8 mile. The people were very nice and it felt so good to be in a true geek crowd. It would be great to attend such gatherings on a permanent basis so I am crossing my fingers hoping that the organizers will pay more attention to San Francisco.

The official presentation was a rather lame discussion of a nascent coding guideline. I would say that this is a non-issue (at least in the scope they mentioned) usually addressed by each team (e.g. questions such as "2-space v 4-space v tab formatting"). The fact that some people come to Scala from fairly uncommon scripting backgrounds complicated the discussion. Later the conversation shifted to real life experiences and favorite tools and that part was more instructive.

It was mentioned that in contrast to Java the question of Scala style is open because different approaches are used in Scala standard library. Many people are disappointed to discover that behind the functional API there is a lot of imperative code. In other words, although Scala seems to be the front runner in Java 2.0 race there is very little infrastructure (both conceptual - such as books/idioms/style guidelines and concrete - such as DI containers) in place or even under active development. And it looks like there is still enough confusion whether to re-use/wrap Java components or re-implement them from scratch.

Actually, I am surprised to find out that non-Java folks are interested in Scala. Judging from blogs web-heads are into dynamic /scripting languages such as Ruby. I believe Scala should be more appealing to people who moved to Java from C++ or find Erlang too weird because of its OOP-unfriendly nature.

Improved Scala support in Idea

The other day I noticed that IntelliJ had issued an updated Scala plugin. Unfortunately it works in Idea 8 only (with a good justification) so I had to install its beta as well. The good news is that it was worth it. I remember the original plugin which did not qualify even as a toy. This time they made real progress.

On the positive side now there is decent syntax highlighting, Ctrl-click navigation and other basic Idea facilities actually working. Clearly there is a lot of space for improvement from better code correctness checks to Scala javadoc support in Ctrl-Q but at least now it is possible to play with the language in a meaningful way.

This year I have seen so many blogs telling about Java decline and the rise of dynamic and/or functional languages that I decided it's time to extend my horizons. I decided to look at Scala (as a "better C++ than Java") and Erlang (because of its telco roots and the availability of a very good book). So far I have been struggling mightily with weird syntax of functional languages.

What I really find puzzling is that despite all the buzz about superiority of dynamic/functional languages I have not seen a single book on design with them. The standard OOA&D cannot be completely applicable (e.g. there are no classes in Erlang and no interfaces in a typical dynamic language) and even UML is likely to be more convenient for pure OO. Taking into account how long and hard it was for most people to master true OOP I am surprised with the lack of such discussions.

Development documentation and Wiki

For me Wiki is a relatively new development documentation media in comparison with Word documents. Although I understand its advantages I am also familiar with its inconveniences. To me Wiki seems to be a compromise where we essentially trade order for universal accessibility.

Recently I had an opportunity to compare the two approaches. Having implemented a new component I wanted to describe it for posterity at the end of the sprint. For simplicity sake I drafted a Word version of the design specification on the basis of a template I came up with a few years ago.

It should be noted that our startup would hardly win any award for the quality of development documentation. In our line of business even established companies struggle with it and fashionable Agile methodologies provided well-intentioned justifications for getting rid of it altogether. So my second goal was to use the opportunity to give a good example of how decent documentation looks like.

I was not surprised to know that although our Director of Engineering appreciated the content he immediately asked me to convert it into Wiki pages. I should admit not all of his objections sound convincing to me. I would even argue many of them are nothing more than a lame excuse for developers who just do not care. Among them were:

• It is hard to update, especially in a collaborative fashion. This means that it might easily become out of date as the system changes.
• It is hard to index and reference. A multi-page wiki document can be easily bookmarked, watched for changes, and referenced from bugs/tasks in the future.
• There is a level of formalism (naming and numbering conventions) that might discourage contributions from people other than the original author.

Just to be balanced here is my Wiki hate list:

• It's impossible to have a document template (analogous to ".dot" files in Word)
• It's impossible to baseline a document together with code in VCS
• It's impossible to version a document as a whole
• Splitting a long document into Wiki pages is painful (you need a page naming convention, it's significantly less convenient to format, there is no support for automatic section numbering at different levels, there is no way to generate a TOC)
• It's difficult to print the whole document
  

In other words, Wiki loses a lot of power in comparison with real documents in exchange for, basically, WWW-like look and feel. From a more practical perspective, there are a few question to answer before you write a Wiki document or migrate from a Word one.

• A dedicated wiki section. You will need a root for the hierarchy of development documents. Virtually all the wikis I have seen were structured by departments at a high level (e.g. Development, Operations, QA). Under Development pages are typically grouped by type rather than subsystem as well. So although theoretically there might be already a sub-tree for each component in your system I would expect to find or start a new tree somewhere under Development (e.g. Development/Development documentation/Component X)

• A standardized tree structure for every component. A conceptual document will need to be split into multiple physical Wiki pages if only to keep them short enough. It' is important to keep a uniform tree structure for all the components.

• A page naming convention. There are a few different page types such as a component root (e.g. "Component X"), a document root (e.g. "Component X/Design specification") and document chapter (e.g. "Component X/Design specification/Static view"). It would be even messier if a chapter were comprised of multiple pages. Now those were just page titles, real page names would be like "componentx", "componentx_sds" and "componentx_sds_static".

• To have access to multiple versions of the same document pages should have versions. Although Wiki keeps track of page changes it is of little convenience if you want just to read a particular final version (as opposed to searching through multiple drafts with highlighted changes). So I would expect all the pages to carry component version numbers as well. Consequently, page names are likely to resemble "componentx_1_0_0", "componentx_1_0_0_sds" and "componentx_1_0_0_sds_static".