The Art of Science

One of the complaints I’ve heard about so many languages is an inability to do certain things from the interactive interpreter which you can do otherwise in content loaded from files.  Erlang does not allow module declarations from the interactive interpreter.  And while it’s not entirely impossible, it’s indicative of a wide gulf between the language as evaluated on the fly and a language which is fundamentally designed to be compiled.

I greatly enjoyed how Ruby allows you do to virtually anything from the interactive interpreter that you otherwise could in normal program code, including defining functions as well as declaring modules and classes.  I sought to do the same in Reia, my dynamic scripting language for Erlang’s VM.  Reia now allows on-the-fly module declarations from anywhere, including both scripts and the interactive interpreter.  For example, the latest version on github allows you to do declare a module from the interactive interperer like this:

  >> module Foo
  ..   def bar
  ..     42
  .. 
  => ~ok
  >> Foo.bar()
  => 42 

This declares a Reia module named ‘Foo’ and defines one function under it, which returns 42.  After being declared, the ‘Foo’ module is compiled to BEAM bytecode and loaded directly into Erlang’s code server.  The "~ok" you see returned (which is Reia syntax for an atom called ‘ok’) is actually coming from the Smerl library by Yariv Sadan, and indicates that the module was successfully compiled and loaded into the code server.  Smerl allows for simple metaprogramming in Erlang by allowing you to build "meta modules" on-the-fly which then outputs all the Erlang forms needed to pass to compile:forms in order to buld a BEAM (or HiPE) module.

When Reia evaluates module declarations, it first compiles the Reia code to Erlang forms then passes them to Smerl to build modules, which invokes the Erlang compiler and loads the resulting bytecode into the Erlang code server.  The end result: on-the-fly module declaration.  You can call methods from compiled Reia modules just as you would any Erlang function, just keeping in mind that the module name is capitalized.  From eshell, you could call ‘Foo’:bar() to invoke the same function.

This allows a number of things which Erlang presently does not, and that doesn’t just mean declaring modules from the interactive interpreter.  Reia allows multiple module declarations from the same file, and furthermore provides a toplevel scope which exists outside of modules and is evaluated as the file is being loaded.  This means you can declare modules in a file then invoke functions in them immediately afterward, just like the above example from the interactive interpreter.

There’s a number of limits at present.  All module declarations need to be done from toplevel scope, i.e. you can’t conditionally declare a module.  You also can’t place statements besides defs inside a module declaration, so you can’t conditionally declare functions/methods either.  However, that said, module declaration is no longer limited to compiled code in Reia.  You can now do it anywhere.

There’s several possible optimizations down the road.  The compiler could be optimized to:

1. Automatically produce HiPE versions of the modules if HiPE is available
2. Keep a cache of generated BEAM/HiPE bytecode and only recompile if the original source file changes
3. Precompile multiple modules in the same source file, employing cross-module optimizations

 

Erlang programmers are awfully vocal about the idea of single assignment: unlike virtually every other language on the planet where you are free to bind values to the same identifier over and over again, in Erlang you can do it once and only once.  However, if you ask an Erlang programmer why multiple assignment is a bad thing you’re unlikely to get a straight answer:

"It is reason why once variable assignment is in Erlang and why it is good think. One advice for you: Don’t change well proved language until you worth know it."

"Are you saying Joe Armstrong is wrong?"

"If I have to explain to you why multiple assignment is bad, you obviously don’t know Erlang well enough"

I was rather relieved to see Damien Katz, one of the principal authors of CouchDB, expressing similar concerns about Erlang’s lack of multiple assignment in his blog post "What Sucks About Erlang":

In C, lets say you have some code:

int f(int x) {
  x = foo(x);
  x = bar(x);
  return baz(x);
}

And you want to add a new step in the function:

int f(int x) {
  x = foo(x);
  x = fab(x);
  x = bar(x);
  return baz(x);
}

Only one line needs editing,

Consider the Erlang equivalent:

f(X) ->
  X1 = foo(X),
  X2 = bar(X1),
  baz(X2).

Now you want to add a new step, which requires editing every variable thereafter:

f(X) ->
  X1 = foo(X),
  X2 = fab(X1),
  X3 = bar(X2),
  baz(X3).

 

Yariv Sadan, the creator of ErlyWeb and perhaps the most prominent Erlang blogger, agreed that he’s run into it before and that it can be annoying.  In his response to Damien Katz, he suggested the following:

If you’re writing code like in Damien’s example and you want to be able to insert lines without changing a bunch of variable names, I have a tip: increment by 10. This will prevent the big cascading variable renamings in most situations. Instead of the original code, write

f(X) ->
  X10 = foo(X),
  X20 = bar(X10),
  baz(X20).

 

then change it as follows when inserting a new line in the middle:

f(X) ->
  X10 = foo(X),
  X15 = fab(X10),
  X20 = bar(X15),
  baz(X20).

 

Yes, I know, it’s not exactly beautiful, but in the rare cases where you need it, it’s a useful trick.

What is this?  BASIC?  Are we’re back to line numbers as a good solution to this problem?  I have the utmost respect for Yariv and am using his excellent smerl library as part of Reia, but that’s a truly silly answer.

I’m going to try to go through some of the more common complaints against multiple assignment and see if I can address them…

Immutability and Multiple Assignment Are NOT a Boolean Decision

Perhaps one of the most common complaints against multiple assignment goes a little something like this:

Immutability is a key feature of a concurrent programming language

Multiple assignment breaks immutability

Therefore, multiple assignment does not belong in a concurrent programming language

Except there’s one thing wrong with that argument: multiple assignment and immutability are two completely different things.  The single assignment cargo cult loves to conflate the two terms.  I cannot tell you how many times I’ve seen someone toss out immutability as a red herring in arguments about multiple assignment.  They are not the same thing and have absolutely nothing to do with each other.

To begin, let’s look at how Erlang’s creator Joe Armstrong explains how to convert a statement which uses multiple assignment into one that does not:

How can you express something like X = X + 1 in Erlang?

The answer is easy. Invent a new variable whose name hasn’t been used before (say X1), and write X1 = X + 1.

Simple as that!  Each time we encounter the potential reassignment of a variable, just pick a new name and bind to that.  At this point you might be wondering: couldn’t the compiler do that for you?  Isn’t it easy to take something like:

X = 42
Y = 1
X = X + 1
Y = Y + X

and transform it into an equivalent set of code which uses single assigment?  Of course it is:

X0 = 42
Y0 = 1
X1 = X0 + 1
Y1 = Y0 + X1

In the new form, each variable is assigned exactly once, with the original variables split into versions.  The compiler can take the original code which uses multiple assignment and convert it to what’s known as static single assignment form.

There’s nothing magic about this.  If the values the variables are bound to are immutable due to language constraints, they remain immutable.  This is just a simple compile-time transformation which has zero effect on the program when it is running.

Repeat after me: immutability and multiple assignment have absolutely nothing to do with each other.  In fact, SSA form has been proven equivalent to the Continuation-Passing Style that was a popular intermediate form for functional languages (functional programmers may appreciate this paper instead).  SSA form is effectively an intermediate lambda calculus representation of languages which allow multiple assignment.  In other words, there exists an intermediate functional representation for imperative programs.

That said, today I began writing a pass for the Reia compiler which converts the original Reia abstract syntax with multiple assignment into SSA form before compiling it to Erlang forms.  In other words, I’m preprocessing a language with multiple assignment and compiling it to an equivalent Erlang program with single assignment.

Pattern Matching and Multiple Assignment Can Coexist Peacefully

Erlang makes extensive use of an idea called pattern matching.  Where the "=" operator in most languages is used exclusively to bind a variable on the left side to a value on the right, in Erlang it’s used to match two expressions.  This means that if a variable is unbound it’s assigned, and if it’s bound it’s matched as part of the pattern.  Patterns are a fundamental construct in Erlang and used as part of things like function declarations and case statements.  Consider the following example (in Erlang): 

X = 42,
Y = 0,
case {42, 24} of
   {Y, Z} -> Y + Z;
   {X, Z} -> X + Z;
   _ -> X + Y
end.

It may be hard for non-Erlang programmers to tell what’s going on there.  Perhaps I can start by telling you that the case statement returns "66", which is 42 + 24. This is because Erlang’s case statement is matching the expression {42, 24} against the three patterns listed in the case statement: the first is {Y, Z}, the next is {X, Z}, and the last is _ which acts as a catch-all pattern.

When Erlang matches a pattern containing variables, if the variable is bound it’s used as a part of the pattern.  In the first case, {Y, Z}, since the variable Y is bound it’s used as part of the pattern, so the first case is effectively {0, Z}.  Since 0 and 42 don’t match, the case statement moves onto the next pattern, {X, Z}, which is effectively {42, Z}.  Here the pattern matches, but since the variable Z isn’t bound the match expression binds it to the value 24.  The result is 66.

Erlang’s pattern matching relies on the idea of single-assignment variables in order to operate.  So how could you have multiple assignment without breaking pattern matching?

Simple: introduce a unary operator for use in patterns which prevents variables from being rebound.  This is precisely how Reia aims to solve the problem.  Here’s the same example as above in a hypothetical "multiple assignment Erlang" which uses a unary * operator to indicate that a variable should not be rebound in a pattern matching expression: 

X = 42,
Y = 0,
case {42, 24} of
   {*Y, Z} -> Y + Z;
   {*X, Z} -> X + Z;
   _ -> X + Y
end.

The result is virtually identical, but here it’s explicitly clear which variables are bound and being used as part of the pattern matching expression, and which ones we wish to bind.  Personally I like this approach better and think it makes it easier to read the pattern matching expression, as opposed to inferring from context which variables are bound and which ones aren’t, but YMMV…

Side Effects Really Just Aren’t That Bad

I can see a typical Erlang programmer actually following the last two arguments and perhaps even conceeding that they have a certain degree of validity.  Here I expect to hit a wall.  You, the hardcore Erlang programmer, will simply not buy this argument, and that’s fine, different strokes for different folks.  But first let me just point out why it’s hypocritical to be a side effect-hating Erlang programmer.

Erlang’s entire concurrency model is built around side effects:

1. When you send a message to another process, you’ve caused a side effect
2. When you register or unregister a process, you’ve caused a side effect
3. When you spawn a process, you’ve caused a side effect
4. When a node connects or disconnects to a distributed Erlang system, it causes a side effect (the value of erlang:nodes/0 changes)

and the list does go on… if you’re an Erlang programmer who thinks side effects are truly bad, perhaps you should be looking at concurrent languages which don’t have side effects at the heart of their operation.

All that said, I will admit: I have encountered errors where a variable has an unexpected value because it was accidently rebound.  I’d say that sort of thing happens to me about once a month, and usually entails about 15 minutes or so of debugging time.  YMMV, but as far as I’m concerned, it really isn’t that bad.

In my case, I’m saying that single assignment would save me about 15 minutes of debugging time per month.  What would I be trading that 15 minutes per month for?  Well consider these examples, the first in Ruby:

x = func(x) if y

and an equivalent expression in Erlang:

X1 = if
  Y -> func(X);
  true -> X
end.

Here Erlang isn’t being helped by its unwieldy if syntax, but that syntax is made even more unwieldy by the lack of multiple assignment.  Here we’re binding a completely new variable, X1, and so the if statement must consider if the Y condition is true and return X after having been run through the given function.  However, we still need to bind X1 to X’s original value in the case that the Y condition is false.

In the Ruby example, we don’t need to worry about the latter case, since it’s implicitly done for us with a side effect.  In the end, single assignment causes more work for the programmer, both in this case and the sorts of examples Damien Katz gave above.

Is it really worth it?  Does having a more verbose language that from a certain perspective is easier to reason about actually save you time in the long run?  My answer is no…

Single Assignment is Weird

In the programming language I’m creating for Erlang’s VM, Reia, I’m allowing multiple assignment by compiling to SSA form.  In making this decision for Reia the other arguments above hardly factored in whatsoever.  There was one overall motivating factor in making the decision: single assignment is weird.

The overwhelming majority of programmers have only used languages with multiple assignment.  Multiple assignment is the de facto standard, and single assignment is one of the things that makes Erlang weird to new programmers.  Even Joe Armstrong conceeds this:

However, if you try to assign a different value to the variable X, you’ll
get a somewhat brutal error message:

4> X = 1234.
=ERROR REPORT==== 11-Sep-2006::20:32:49 ===
Error in process <0.31.0> with exit value:
{{badmatch,1234},[{erl_eval,expr,3}]}
** exited: {{badmatch,1234},[{erl_eval,expr,3}]} **

What on Earth is going on here?

This is practially the very first thing Joe must introduce in his book.  If you hand Erlang to a novice, "what on Earth is going on here" will precisely describe the discovery of Erlang’s single assignment.

In Reia, I’m trying to keep the "what on Earth is going on here" feeling to a minimum.

It takes a particularly crazy individual to create a programming language, to the point that many of them are indistinguishable from serial killers.  I’m apparently in the extreme minority of people who want to create a programming language but eschews the serial killer stylings.  My language is called Reia, and there’s plenty of information about it available on Reia’s wiki.  But I’m not going to talk about Reia itself quite yet…

Given that most programming language creators don’t care enough about their own visual identity enough to distinguish themselves from serial killers, it’s not too surprising that the visual identity of a programming language goes ignored by them as well.  When Guido van Rossum (a man some might confuse with Jeffrey Dahmer) was creating Python, the visual identity he had in mind probably looked something like this:

However, once the community got ahold of Python’s visual identity, the implicit "Monty" part of the language’s identity was lost, and instead we end up with the new official Python logo:

Ruby is a language with a language creator apathetic to its visual identity, but it has attracted a number of web designers, and wound up with this logo:

Here we see a Ruby with a completely inexplicable diamond cut.  Compare this to the sort of cut you’d actually see on real rubies:

Well, that’s it.  I’m taking a stand.  I’m not going to let others decide Reia’s visual identity.  Here it is:

The Reia visual identity guidelines are as follows: The word "Reia" in the Futura typeface, preferably in black. That’s it.

Futura is a typeface intended to express modern models, rather than be a revival of a previous design. Futura’s appearance is intended to impart the ideas of efficiency and forwardness, ideas I hope Reia expresses in its design.

No snakes or gemstones, please…

What a confusing title for this blog post.  I’m certain the programmer community who is reading this can guess what it means.  No, it’s not about humans typing on keyboards, stereotyping, or any of that.  It seems to me that most programming languages enforce certain constraints on their users which aren’t really in line with how humans operate.  I think type systems can be human-sensitive, if designed properly…

The important point is to look at how people reason about problems.  I don’t know about you, but I try to look for what similar problems can relate to the problem I’m presently experiencing, and synthesize that knowledge into a solution that works for me now.  I look for similar solutions to similar problems.  Why can’t programming languages work that way?

Within the traditional statically typed object oriented languages, there’s sort of an obsessive compulsive approrach to solving similar problems: you use inheritence.  CaringForCats is similar to CaringForDogs, right?  I mean, they’re both mammals.  So let’s turn CaringForDogs into a CaringForMammals base class, move the dog-specific behavior into a new CaringForDogs subclass of CaringForMammals (which we abstracted from the original CaringForDogs class) then subclass CaringForCats from that.  I mean, dogs are a mammal, therefore we can abstract our experience with dogs to all mammals then subclass our relationship with cats from that, right?

Unfortunately, that’s not how it works in reality.  Caring for cats is nothing like caring for dogs, even though we’ve made the observation that they’re both mammals.  Despite our ability to classify dogs and cats as both mammals, we’re overlooking the reality of the situation: caring for dogs is nothing like cats, so why are we even trying to make an "abstract" mammals base class when really we’re dealing with two different types altogether?

The thing is: some things that work for dogs works for cats.  There’s certain aspects of the dog experience which translate over.  Maybe not enough to say they’re entirely related, but maybe we can learn what dog things work for cats, then go from there.

So, how do you relate that back to programming?  Well, we have two things which share a minimal degree of similar interfaces, but the dissimilarities are enough that it’s really silly to say that CaringForCats is a subset of CaringForMammals… there really is no general way to care for mammals, but there maybe a few tricks which carry across, to the point you can try to care for a cat as if it were a dog and work your way from there, specifically learning what doesn’t work, simply through trial and error.

Dynamic languages have introduced the idea of "duck typing"… metaphorically, if something behaves like a duck, then treat it like a duck, and things will more or less work out.  Sorry, that’s if it quacks like a duck.  But nothing quacks like a duck except ducks.  So what happens when you try to quack to a cat?

Duck typing tries to cast off the uniformity of abstract base classes by letting you experiment with trial and error and find your own feel for an object which is similar, but expresses many different qualities from what you know.  Mammals don’t quack like ducks, but you perhaps have past experience with the differences between mammals and ducks, and specifically you have experience with dogs.  That doesn’t mean you’ll know how to deal with cats, but you can learn, and some of your experiences with dogs will translate over into cats.  There’s no rigid, formally defined structure relating cats and dogs, but as a human, you can do your best with your past knowledge and try to translate dogthink into catthink.

Bottom line, I think duck types are a powerful, human-friendly concept which is easily written off by the static typing OO purists as some form of impure abstract base class.  Java pursued this approach with Interfaces, which choded the programmer with complaints about not implementing the entire interface definition, even if certain parts didin’t apply.  Even the smartest programmer can’t know beorehand all the ways a particular interface can be abstracted… if you can get 90% of the way there, and leave the remaining incompatibilities for the API user to fix, haven’t you accomplished enough?  By leaving incompatibilities in place, you don’t have to abstract away the best underlying features of some system simply for the sake of compatiblity.

So really, why is it called duck typing?  I think, being as in-tune with the way humans accomplish tasks as it is, it should be called human typing…

I’ve been a fan of Ruby for awhile.  It’s a great language, but the implementation of Ruby is pretty terrible.  This was made painfully clear by some recent security vulnerabilities and the way they were handled by the Ruby core team.  The Ruby implementers didn’t disclose the exact nature of the vulnerabilities nor did they disclose the steps they’d taken to correct them.  This leaves people looking at the source as our only way of knowing what actually happened.

Now, this would all be bad enough on its own.  However, the Ruby release process has recently been plagued by massive amounts of changes combined with a painful lack of testing.  The patches fixing the security vulnerabilities were applied to the current development codebase of various versions.  There’s Ruby 1.9, the unstable version of Ruby with massive changes that break backwards compatibility.  There’s Ruby 1.8.7, which contains many changes backported from Ruby 1.9 which break backwards compatibility and cause various random errors as the initial release contained many bugs.  And then there’s 1.8.6, a version which remains mostly compatible with versions of Ruby most current code was developed on.  However, 1.8.6 is an evolving release consisting of many different patchlevels.  And as it happens, at some point someone committed a buggy change to the Ruby 1.8.6 branch which makes the Ruby interpreter crash when running code using the Rails web framework, which remains Ruby’s most popular use case.

The security patch fixing the vunerabilities was applied to Ruby 1.8.6 after the buggy change was introduced.  This leaves you with some painful choices: run a version of Ruby with known security vulnerabilities, install a version which crashes, or update your application to be Ruby 1.8.7 compatible.  This isn’t a good situation to be in.  This is further compounded by the fact that the only version of Rails which runs on Ruby 1.8.7 is Rails 2.1, which also introduces changes which aren’t backwards compatible.

Things like this really make me wonder if the Ruby core team cares at all about the people using their language.

The worst part is many, many people have offered to help improve the Ruby development process.  The Rubyspec team, who have painstakingly specified the Ruby implementation using the excellent RSpec utility, offered to implement automated testing of all commits to the Ruby repository with continuous integration (i.e. every commit is checked for failing specifications).  The core team refused.

Unfortunately, I have no hopes for the Ruby development process improving any time soon.