This is the sixth of a series of articles on what I’ve learned about Erlang (and Elixir) from writing Erl2ex, an Erlang-to-Elixir transpiler. This week we study guard clauses and learn why Elixir guards look the way they do.
Update (2016-06-11) … Jose Valim pointed out that it is possible to recreate guard sequences in Elixir after all, by using multiple “when” clauses. I updated the relevant section. Thanks for the correction, Jose!
A guarded question
In the Elixir standard library docs, you’ll occasionally come across the following notation:
“Allowed in guard tests.”
If you’ve worked much with Elixir, you probably know what that refers to. Guards, which you can recognize as
when clauses in Elixir, can contain only certain types of expressions. You can compare values and test types, using functions that are “allowed in guard tests”:
However, most functions, including those you define in your own modules, are not allowed in guards:
Why the limitation? Is there no way to allow user-defined functions and other useful constructs in guards?
These are questions that have been asked repeatedly in both the Erlang and Elixir communities. One answer that you may have heard is that guards cannot produce side effects. But why is that important? What is a guard really doing?
To answer these questions, we’ll look a little deeper into how Erlang defines guards. First we’ll examine the structure of Erlang guards in comparison with Elixir guards. We’ll explore differences in how the two languages handle exceptions in guards, as well as differences in the way conditionals and comprehensions treat guards. Finally, we’ll talk about why the limitations are there, and what you can do to get around them. This will be a fairly long article, but there’s a lot of useful information to glean, so let’s get started.
Erlang’s complex guards
Erlang guards have a somewhat more complex structure than what we usually find in Elixir. In Elixir, a guard is simply a boolean expression. But in Erlang, it may comprise more than one expression; in fact, generally, it is actually a list of lists of boolean expressions.
Let’s start with a simple guard in Erlang:
In this function clause, the expression
is_integer(X) is called a “guard expression”. But it is only a simple case of a guard. You can add more guard expressions, delimited by commas, and this clause will be selected only if all are true. In other words, it behaves almost as if the expressions were joined into a single expression by Erlang’s “andalso” operator. (I say “almost” because there is a subtle difference that I’ll discuss later.)
The two comma-separated boolean expressions must both pass for this function clause to execute. That is, the above is (almost) equivalent to:
Such a list of comma-separated guard expressions is simply called a “guard”. But Erlang doesn’t stop there. You can also add additional guards (i.e. lists of guard expressions), delimited by semicolons. The function clause will be selected if at least one of those guards passes. In other words, multiple semicolon-delimited guards behaves almost as if they were joined by the “orelse” operator. Such a list of semicolon-delimited guards is known as a “guard sequence”.
This is (almost) equivalent to:
At first glance, the commas and semicolons may look simply like shorthand for boolean operators. It turns out there is a difference, but before we get into that, we should note that the same restrictions apply to expressions in Erlang guards as to Elixir guards. Only certain operators, and a few blessed built-in functions, are allowed in Erlang guard sequences. Anything else is rejected by the compiler.
Again, why the restriction? The reason is as revealing as it is subtle. Guards may look like normal code, but they are not. They are actually something else. Ulf Wiger puts it, I think, very helpfully:
Think of guards as an extension of the pattern matching, i.e. a purely declarative, and quite pervasive, part of the language.
Looking at it this way, it could be argued that the problem is that some parts of the patterns look like normal expressions. ;-)
I think Erlang, with its “complex” guard sequences, helps us to see this more clearly than Elixir guards by themselves. In Elixir, a guard looks just like any other expression. But Erlang guard sequence syntax is different from a “normal” Erlang expression. It can include comma- and semicolon-delimited lists of expressions, which can’t normally appear in other places:
So Erlang helps us understand that guards are actually something else; they are better considered part of the pattern match rather than a normal expression. And just like you can’t put arbitrary code (like function calls) in a pattern match, because it just doesn’t make sense as part of pattern match syntax, neither can you include arbitrary code in a guard.
Specifically, code that has side effects (which, broadly speaking, means code that could send and receive messages) is not allowed in a guard, as is code that could fail to terminate. (As a corollary, guard evaluation is not Turing-complete.)
And if you think about it, such restrictions make a lot of sense. Suppose you have a function with a number of pattern-matched (and guarded) clauses. If those guards were allowed to have side effects, then you would need to be able to control whether and in what order they execute. Programmers do not tolerate non-deterministic side effects. Erlang would need to specify and lock down those semantics, and that would hamstring the function dispatch. For example, it would prevent the VM from reordering, parallelizing, caching, or omitting guard execution, thus cripping the optimization of the language’s most crucial bottleneck. So the ban on side effects is critical to the viability of guards as a feature.
But let’s come back to the question of why Erlang has guard sequences at all. Why not simply support boolean expressions joined with “ands” and “ors”?
Suppose you had a function that took a list but needed to behave differently if it had a length of 2. The built-in
length function could be used in a guard for this purpose.
This function behaves how we’d expect.
length function requires a list. What happens if you pass in something else, like a binary?
length function crashes if you pass in something that is not a list. However, interestingly, it doesn’t crash when used in a guard. That’s because any “error” that is raised in a guard, merely causes that test to fail (i.e. evaluate to false). This behavior is the same in both Erlang and Elixir. Guards never crash; they only succeed or fail.
Why? Again, it makes sense if you think about it. A crash is like a side effect. If guards could crash, you’d have to specify and control whether and in what order they execute, so that you know what crash should take place. Again, that would hamstring function dispatch and prevent the VM from optimizing it. So instead, the BEAM turns guards into pure functions by removing the possibility of crashes.
So far so good. But now let’s extend our function to handle binaries in addition to lists. And again, we want inputs with a length of 2 to follow one code path, and other inputs to follow another. In Erlang, we can extend the guard sequence with another expression to test binary size:
Remember, a guard sequence passes if at most one of its parts passes. In this case, the first clause will match if a list of length two, OR a binary of two bytes, is passed in. In the latter case, the first expression
length(X) == 2 causes an error, but since it is in a guard, it just evaluates to false and moves on to the next expression,
byte_size(X) == 2, which succeeds. Let’s test it:
Remember how we said that a guard sequence (i.e. the semicolon delimiter) is almost but not quite the same as using the “orelse” operator in Erlang? Now we’ll see why. In the above example, when you pass in a binary, the
length call throws an error, causing that guard expression to fail, but the rest of the expressions in the guard sequence still have a chance to execute. However, if you use “orelse”, then both the
byte_size calls are in a single expression:
Now if you pass in a binary, the
length function throws an error, which causes the entire expression to fail.
In other words, Erlang’s guard sequences let you “insulate” individual expressions from errors thrown by other expressions.
What about Elixir? As in Erlang, a crash in a guard will cause the entire guard to fail. So attempting to check either list length or binary length using the “or” operator behaves the same as when we tried to use Erlang’s “orelse”.
Again, if you pass in a binary, the
length function throws an error, causing the entire expression to fail:
So in Elixir, are we stuck? Not quite. It doesn’t seem to be well-documented, but Jose helpfully pointed out that if you supply multiple “when” clauses, Elixir will treat it as a guard sequence, just as if you used semicolon delimiters in Erlang.
Now it works! If the first guard fails with an error, the second will still run, and return the result we want:
Alternatively, you can test the types of inputs in your guards. Indeed, perhaps it’s good coding practice to be explicit rather than depending on the exception-suppressing behavior of guards.
But that isn’t the only way guards are treated differently between Elixir and Erlang.
The meaning of “if”
Those of us who came from imperative and object-oriented languages tend to have a very concrete expectation around how conditionals, such as the “if” statement, should behave. In Elixir, because of pattern matching, the “if” macro is not used as frequently as in other languages, but it still feels very familiar and follows the same traditional behavior.
It can be surprising, then, how different Erlang’s “if” statement is. To start off, we might know that, while Elixir’s version is a binary conditional (choosing one of two branches based on a boolean result), Erlang’s version is an n-ary conditional that may depend on any number of expressions.
At first glance, this looks like it simply corresponds to Elixir’s
However, there’s an important difference: Elixir’s
cond supports arbitrary expressions, but Erlang’s
if actually uses guard sequences. This means:
- Expressions in Erlang’s
ifstatement are limited to those that can appear in guards. In Elixir’s
cond, you can include arbitrary expressions with side effects.
- You can include multiple comma- and semicolon-delimited expressions (i.e. guard sequences) in Erlang’s
condrequires you to use
oroperators for this purpose.
ifstatement exhibits the exception-suprressing behavior of guards. Elixir’s
So Erlang would allow us to expand this function to support binaries:
As we saw earlier, guards suppress exceptions, and furthermore, the expressions in each guard sequence are insulated from errors thrown by the others. So we could pass a binary into the above function, and it would work as expected. Reproducing this functionality using Elixir’s
cond would be more difficult. We would need to check the type explicitly because otherwise exceptions would get thrown normally.
Finally, it is also important to remember the difference between guards and Elixir expressions with relation to truthiness. In a guard, the atom
:true is considered true, and all other values are considered false. In an Elixir expression,
:false are considered false, and all other values are considered true.
Overall, whereas Elixir’s
cond macros behave very much like “traditional” conditionals, Erlang’s
if statement is best understood through its connection with guards. It is a way to invoke guards and their peculiar semantics without needing to create a new function. On the Elixir side, if you want to achieve that same goal—specifically invoking guard semantics—you might consider the
But wait… there’s more!
Both Elixir and Erlang support comprehensions. These are very useful, highly expressive ways to generate and filter collections of data. But the implementations in the two languages are subtly different in several ways. One of those is, you guessed it, the use of guards.
In Elixir, the “filters” in a comprehension are arbitrary expressions. They may produce side effects and throw exceptions. In Erlang, they may be arbitrary expressions or they may be guards. This can get confusing, so let’s look at an example.
This Erlang function takes a list of lists as input. It returns the elements that are of unit length, and discards the rest. A comprehension makes this very simple:
You could write this similarly in Elixir:
These two functions behave similarly if you give them the expected list of lists:
However, suppose one of the elements is not a list. As we’ve seen, the built-in
length function throws an error if given a non-list argument. Here are the consequences:
The Elixir version crashes because our comprehension tries to evaluate
:non_list, which throws an error. However, in the Erlang comprehension, the filter is treated as a guard. The error is suppressed, and simply causes that filter to return false.
Does this mean filters in Erlang comprehensions are simply guards, just like the filters in Erlang’s
if statement? Not quite. It turns out that when Erlang compiles a comprehension, it analyzes each filter, treating it as a guard if it is a valid guard epxression, or otherwise treating it as a normal expression. So, recalling our Erlang comprehension:
Currently, that length check is a valid guard, and so Erlang compiles it into a guard. But we can force it to be a normal expression by introducing syntax that is not allowed in a guard, such as a local function call.
When we execute that new function, it now behaves like our Elixir version. The expression is no longer a guard, so it no longer suppresses the error.
So in Erlang, a comprehension may or may not include guards, depending on what kind of code is present. But in Elixir, you have no choice: comprehension expressions are always normal expressions rather than guards. Personally, in this case, I prefer Elixir’s approach. It’s consistent, and offers less chance of confusion.
Let’s wrap up by briefly discussing another Elixir technique for dealing with guards. We’ve seen that the compiler conservatively limits what can appear in a guard, whitelisting a small set of known safe functions and disallowing everything else, because it wants to guarantee the lack of side effects. In particular, you cannot define your own functions and call them from a guard. This makes it difficult to write complex guards (which, one could argue, is the point). However, in some cases, you can get around this in Elixir by using a macro instead of a function.
Suppose we were writing a function, such as “zip”, which wants to behave specially if the lists passed as arguments are the same length. We could write a function to test two such lists:
Unfortunately, we cannot use our own functions in a guard.
Instead, let’s rewrite same_length as a macro.
What’s the difference here? Instead of defining a function, we’ve created a macro that looks like a function but expands at compile time. Effectively, the body of the macro,
length(list1) == length(list2) gets inlined at the call point. This means:
Now the code is legal for a guard, and thus it compiles successfully.
Of course, there are still limitations. You cannot include arbitrary code in such a macro; it must still expand to code that is allowed in a guard clause. Recursion is still out, as are sends and receives. But this technique can sometimes be used to make commonly-used guard expressions more readable.
In fact. this technique is used in several places in the Elixir standard library to define custom “functions” that are allowed in guard clauses. An example is
Kernel.is_nil/1, which could be written trivially as a function, but is implemented as a macro specifically so it can appear in guards. So if you’re looking through the standared library documentation and come across macros that look like they should be simple functions, check for that no-longer-so-mysterious comment, “Allowed in guard clauses”. It might give you a clue as to why the standard library looks the way it does.
Where to go from here
When I was first learning Elixir, guards seemed to be confusingly at odds with much of the rest of the language. It was only through learning Erlang that it became more clear what a guard actually is and how they should be used. I found the relevant chapter in the book Learn You Some Erlang For Great Good to be particularly helpful on this topic.
Elixir by itself obscures some of the distinctive properties of guards because they look nearly indistinguishable from “normal” expressions. To its credit, Elixir drops some features that make guards confusing, such as the option to use them in comprehension filters. However, it is still useful to study how they work in both languages, to avoid being caught by surprise in one of the many corner cases. Incidentally, treatment of guards remains a major source of headaches (and bugs) in Erl2ex, because of the lack of Elixir support for some of those cases.
Next time, we’ll sample a few “missing features” present in Erlang but not supported in Elixir, and look at some workarounds. Feel free to browse the index of articles in this series, and stay tuned for more on Erlang and Elixir’s family ties.