Learning Elixir's with

The upcoming release of Elixir 1.2 will introduce a new special form, with. I wanted to use this post to explore how this new feature works and how to use it effectively going forward.

The purpose of with is to help chain together commands that return different structured results. One particular use case is to make handling errors a little clearer. When a command’s return value fails to match the with clause then it falls out of the entire expression. The examples below should help to make this clearer.

Before we dive into the examples, I should note that I’m running against Elixir’s master branch which has support for the with special form. But, master is closing in on the 1.2 release so you’ll be able to pick up this feature in a stable release soon.

Examples from the Docs

First, I want to make sure that I have a working environment with master including with. As a sanity check I’ll start by copying the examples from the Elixir docs into ExUnit tests.

Here’s the first test:

test "Example 1" do
  opts = %{width: 10, height: 15}
  assert {:ok, 150} ==
    with {:ok, width} <- Map.fetch(opts, :width),
         {:ok, height} <- Map.fetch(opts, :height),
      do: {:ok, width * height}
end

Here with will attempt to match two clauses (the two lines containing <-). Given opts, both matches will succeed, the do block will be executed and its result will be returned. So, the result of the with expression is {:ok, 150} which means that our assertion is true and the test passes.

test "Example 2" do
  opts = %{width: 10}
  assert :error ==
    with {:ok, width} <- Map.fetch(opts, :width),
         {:ok, height} <- Map.fetch(opts, :height),
      do: {:ok, width * height}
end

This is a modified version of the previous example. We have the same with expression from Example 1 but different data. In this case there is no :height key/value pair in opts.

Because there is no :height key Map.fetch/2 will return :error and with will fail to match the second clause. When with encounters a clause that doesn’t match it stops evaluation and returns the right hand unmatched value. That means in this case it returns :error. Hence, our assertion is true and the test passes.

test "Example 3" do
  width = nil
  opts = %{width: 10, height: 15}
  assert {:ok, 300} ==
    with {:ok, width} <- Map.fetch(opts, :width),
         double_width = width * 2,
         {:ok, height} <- Map.fetch(opts, :height),
      do: {:ok, double_width * height}
  assert width == nil
end

The final example from the docs shows the scoping rules used in with expressions.

The matching behavior is the same as Example 1 but the clauses compute additional values.

We can see that the first clause computes double_width and that double_width is in-scope during the do block.

We can also see from this example that a variable named width is bound in the first match clause. But, this is not the same variable width in the scope outside of the the with expression. That is, variables bound inside the with expression don’t leak into the outer scope. This is the same as the way the for special form behaves.

Survey of the RFC

The with functionality was proposed in the Introducing with RFC. I’ll summarize the RFC here to …

with is different than for in that it matches on values rather than values from collections. There are simple examples and some larger examples. One example I really liked gave a suggested use case for simplifying nested case statements like this:

case File.read(path) do
  {:ok, binary} ->
    case :beam_lib.chunks(binary, :abstract_code) do
      {:ok, data} ->
        {:ok, wrap(data)}
      error ->
        error
    end
  error ->
    error
end

into something like this:

with {:ok, binary} <- File.read(path),
     {:ok, data} <- :beam_lib.chunks(binary, :abstract_code),
     do: {:ok, wrap(data)}

I find the with version much more succinct and easy to follow. The nested case version is forced to include a lot of extra and repeated lines for error handling that obscures the intent.

Convert existing error handling code to `with`

I started paying attention to the with syntax after searching for good ways to handle errors in Elixir. I found error checking would break up nice pipelines. One way I found was to build monads for describing the error values and then mapping over them.

In a Phoenix based project I’m working on I choose to use a simpler solution. I wrote separate function heads for each of the functions in the chain so that they can pass errors through to the end of the pipeline. This is simple, but can be a bit tedious and the extra function heads obscure the intent a little.

Here’s the code I have:

defp results(conn, search_params) do
  conn.assigns.current_user
  |> Role.scope(can_view: Service)
  |> within(search_params)
  |> all
  |> preload(:user)
end

defp within(query, %{"distance" => ""}), do: {:ok, query}
defp within(query, %{"distance" => x, "location" => l}) do
  {dist, _} = Float.parse(x)
  Service.within(query, dist, :miles, l)
end
defp within(query, _), do: {:ok, query}

defp all({:error, _} = result), do: result
defp all({:ok, query}), do: {:ok, Repo.all(query)}

defp preload({:error, _} = result, _), do: result
defp preload({:ok, enum}, field) do
  {:ok, Repo.preload(enum, field)}
end

I should give a little context.

The results function is a composition of Ecto queries. It starts with my User model and queries the services that the user is allowed to view. Then, it limits those services to the set within a given distance of a specified location. This is based on the form parameters. Then, it fetches the services and preloads the users.

The within function is a wrapper for a function in Service. It handles two cases that don’t require searching for a distance. If a distance search is required it hands off the work of building the query to Service.within. This is the function that can fail.

The all function is part of the error handling case. If within returned {:error, _} then all just passes through that error. If within returns an :ok value then all queries the Repo.

The preload function is similar to all. It passes through errors, otherwise it calls Repo.preload.

Using with I can remove the error cases:

defp results(conn, search_params) do
  with user <- conn.assigns.current_user,
       query <- Role.scope(user, can_view: Orthrus.Service),
       {:ok, query} <- within(query, search_params),
       query <- all(query),
    do: {:ok, preload(query, :user)}
end

defp within(query, %{"distance" => ""}), do: {:ok, query}
defp within(query, %{"distance" => x, "location" => l}) do
  {dist, _} = Float.parse(x)
  Service.within(query, dist, :miles, l)
end
defp within(query, _), do: {:ok, query}

defp all(query), do: Repo.all(query)
defp preload(enum, field) do {:ok, Repo.preload(enum, field)}

Improvements:

No longer need extra function heads just for passing errors through to the end
No longer need to pass {:ok, term} form into these functions, with extracted the values.
No longer need to wrap result in {"ok, term"} from all. This was just for passthrough

I do need to wrap the result of Repo.preload in {:ok, term} because it becomes the return value of results/2. To remove this I would need to do something with the callers of results/2

Actually, I can take this simplification a step further, there is no reason to wrap Repo.all anymore at this point. So I can remove the all function. Also, I can remove the preload function by building the tuple in the do block.

defp results(conn, search_params) do
  with user <- conn.assigns.current_user,
       query <- Role.scope(user, can_view: Orthrus.Service),
       {:ok, query} <- within(query, search_params),
       query <- Repo.all(query),
    do: {:ok, Repo.preload(query, :user)}
end

defp within(query, %{"distance" => ""}), do: {:ok, query}
defp within(query, %{"distance" => x, "location" => l}) do
  {dist, _} = Float.parse(x)
  Service.within(query, dist, :miles, l)
end
defp within(query, _), do: {:ok, query}

I guess there is one more improvement I can make though it doesn’t have anything to do with with:

defp results(conn, search_params) do
  with user <- conn.assigns.current_user,
       query <- Role.scope(user, can_view: Orthrus.Service),
       {:ok, query} <- within(query, search_params),
       query <- Repo.all(query),
    do: {:ok, Repo.preload(query, :user)}
end

defp within(query, %{"distance" => x, "location" => l}) do
  {dist, _} = Float.parse(x)
  Service.within(query, dist, :miles, l)
end
defp within(query, _), do: {:ok, query}

I’ve removed the first within/1 clause because it was redundant with the last clause.

At this point, I’m quite happy with the result. The code is so much smaller and in my opinion the intent is much easier to follow.

Conclusion

In this post we read about and played around with the new with special form. We looked at several examples that explain the feature and its benefits. Then, we used with to refactor some existing code into a clearer and more concise form.

I have to say, I’m very excited about the new feature. I intend to upgrade my projects to Elixir 1.2 and take advantage of with to better handle errors in chains of commands.

Joseph Kain

Learning Elixir's with

Examples from the Docs

Survey of the RFC

Convert existing error handling code to with

Conclusion

You might also enjoy (View all posts)

Convert existing error handling code to `with`