Elixir YAML Parsing

I’ve been writing about using Dialyzer with Elixir lately. But, I’ve started working on a new side project and had a need to parse YAML files. So, this is a post of opportunity in which I’ll share my experineces learning to use yamerl. Expect more such posts in the future as I work on this new project.

Yamerl is an Erlang module for parsing YAML files. I didn’t find any good resources on using yamerl with Elixir. So I’m writing up what I’ve learned in this post. yamerl is available on github and has a README that gives good instructions for Erlang. I’ll try to translate some of that into Elixr.

I wrote this post about a week ago and after writing it I cam across Parsing YAML in Elixir from Alexander Panek. The post goes into good detail on both yamerl and yamler. Definitely worth a read.

Installation:

The instructions say to use rebar but my Elixir project uses mix. Fortunately, mix has support for rebar based projects and it works with yamerl. I was able to simply include yamerl as a github based dependency:

defp deps do
  [
    {:yamerl, github: "yakaz/yamerl"}
  ]
end

and run

mix deps.get
mix deps.compile

Setup

The yamerl process needs to be started. In an Elixir / Mix application this is as simple as adding yamerl to the applications section of mix.exs:

def application do
  [applications: [:logger, :yamerl]]
end

Parsing a YAML file in Elixir

The yamerl README has the following test YAML data

# applications.yaml
- application: kernel
  version:     2.15.3
  path:        /usr/local/lib/erlang/lib/kernel-2.15.3
- application: stdlib
  version:     1.18.3
  path:        /usr/local/lib/erlang/lib/stdlib-1.18.3
- application: sasl
  version:     2.2.1
  path:        /usr/local/lib/erlang/lib/sasl-2.2.1

I’ve saved this data in a file called applications.yaml so I can test load it. Then I started up iex

$ iex -S mix

In iex I loaded up the YAML file like this:

iex(1)> :yamerl_constr.file("applications.yaml")
[[[{'application', 'kernel'}, {'version', '2.15.3'},
   {'path', '/usr/local/lib/erlang/lib/kernel-2.15.3'}],
  [{'application', 'stdlib'}, {'version', '1.18.3'},
   {'path', '/usr/local/lib/erlang/lib/stdlib-1.18.3'}],
  [{'application', 'sasl'}, {'version', '2.2.1'},
   {'path', '/usr/local/lib/erlang/lib/sasl-2.2.1'}]]]

Extracting the data

First of all we need to know how the parsed data is formatted. This is a direct quote from the README:

% List of documents; again, only one document here.
[
 % List of mappings.
 [
  % Mapping, represented as a proplist: each entry has the form {Key, Value}.
  [
   {"application", "kernel"},
   {"version", "2.15.3"},
   {"path", "/usr/local/lib/erlang/lib/kernel-2.15.3"}
  ], [
   {"application", "stdlib"},
   {"version", "1.18.3"},
   {"path", "/usr/local/lib/erlang/lib/stdlib-1.18.3"}
  ], [
   {"application", "sasl"},
   {"version", "2.2.1"},
   {"path", "/usr/local/lib/erlang/lib/sasl-2.2.1"}
  ]
 ]
]

Ok, this is very general especially in that it provides support for multiple documents. For my needs (just like the example), I have only one document. I can extract the first document from the list like this:

iex(10)> [ document | _ ] = :yamerl_constr.file("applications.yaml")
# Entire structure omitted
iex(11)> document
[[{'application', 'kernel'}, {'version', '2.15.3'},
  {'path', '/usr/local/lib/erlang/lib/kernel-2.15.3'}],
 [{'application', 'stdlib'}, {'version', '1.18.3'},
  {'path', '/usr/local/lib/erlang/lib/stdlib-1.18.3'}],
 [{'application', 'sasl'}, {'version', '2.2.1'},
  {'path', '/usr/local/lib/erlang/lib/sasl-2.2.1'}]]

Now, I’ll look at the first mapping which coresponds to the kernel:

iex(12)> [ kernel | _ ] = document
# Entire document omitted
iex(13)> kernel
[{'application', 'kernel'}, {'version', '2.15.3'},
 {'path', '/usr/local/lib/erlang/lib/kernel-2.15.3'}]

Before I went back and read the README carefully I thought this was a Keyword list but it turns out that it isn’t. I was very confused why the Keyword module couldn’t look up any of the values in the data structure. After I went back and read the README more diligently I realized that the data structure is an Erlang proplist which can be accessed with the :proplists module like this:

iex(14)> :proplists.get_value('version', kernel)
'2.15.3'
iex(15)> :proplists.get_value('path', kernel)
'/usr/local/lib/erlang/lib/kernel-2.15.3'
iex(16)> :proplists.get_value('application', kernel)
'kernel'

It is worth noting that the keys and values in the proplist are stored as Erlang style strings which are given the type char_list in Elixir. To convert the values to Elixir String we can use List.to_string/1 like this:

iex(17)> :proplists.get_value('version', kernel) |> List.to_string
"2.15.3"
iex(18)> :proplists.get_value('path', kernel) |> List.to_string
"/usr/local/lib/erlang/lib/kernel-2.15.3"
iex(19)> :proplists.get_value('application', kernel) |> List.to_string
"kernel"

In my case I’ll be using literals for the keys so I can just use single quotes to make char_list literals. But, if you have String variables in Elixir and want to use them as keys when getting values from the proplist then you can use String.to_char_list/1 to convert.