Getting started with JSON Decoding in Elm

This post was first published on ElmPlayground.com but has now been updated and
moved to this blog.
Something that continually trips beginners up in Elm is dealing with JSON
responses from a third party API. I think this is because it’s a completely new
concept to those picking up Elm from JavaScript. It certainly took me a long
time to get comfortable with Elm.
Today, we’ll look at using JSON decoders in Elm to deal with data from an API.
I’ve purposefully made some of the data awkward to show some of the more complex
parts of decoding JSON. Hopefully the APIs you’re working with are much better
than my fake one, but this post should have you covered if not!
Before we get into that though, let’s go through the basics of Elm decoders.
What is an Elm JSON decoder?
A decoder is a function that can take a piece of JSON and decode it into an Elm
value, with a type that matches a type that Elm knows about. For example, if we
have this JSON:
{ “name": "Jack" }

Then I need to tell Elm that the value at the name field is a string, so it
can parse the JSON value "Jack" and turn it into the Elm string "Jack". Elm
ships with many decoders for all of the built in types in Elm, and also the
ability for us to make our own decoders, which is of more interest to us, as
more often than not you’ll be taking an object and converting it into an Elm
record.
Layering decoders
The real power of Elm’s decoders, which is also why they can be pretty
complicated to work with, is that you can combine them to make other decoders.
This is something Brian Hicks wrote about in his
post on Elm decoders being like Lego,
which I highly recommend reading. For example, Elm ships with a decoder for
decoding an object with one field, called JSON.Decode.map. Its type signature
is:
map: (a -> value) -> Decoder a -> Decoder value

What’s important to remember is that all these decoder functions return new
decoders. You have to layer the decoders together to match your JSON. In the
case of map, its arguments are as follows:

(a -> value) a function that will take the decoded value, and should return
data of the type value, which is the Elm data you want to get out of your
JSON.
Decoder a is a decoder that can decode the given JSON and pull out a value
of type a, which will be passed into the function given as the first
argument.

For example, taking the JSON that we had earlier:
{ "name": "Jack" }

Let’s say we want to decode this into the following Elm record:
{ name = "Jack" }

The first step is to create our decoder. We’re going to use map, because we
want to decode a JSON object where we only care about one field. The JSON we’re
decoding could have any number of fields, but we use map because we only
care about one field.
Note: through the following code examples I’ve imported the JSON decoding
module as import Json.Decode as Decode, so I’ll refer to functions as
Decode.map, Decode.string, and so on.
First I’ll define my decoder. The first argument is an object that takes the
decoded value and turns it into the thing I want to end up with. The second is a
decoder that can take a value at a particular field, and decode it. To do that I
use Decode.at, which plucks an item out of the object and applies the given
decoder to it:
userDecoder =
map (\name -> { name = name })
(Decode.at ["name"] Decode.string)

Before we go on, can you guess what the type of userDecoder is here?
It is:
userDecoder : Decode.Decoder { name : String }

Because it’s a decoder that returns an Elm record with a name property of type
String.
Now let’s run this decoder and see what we get. We can run a decoder using
Decode.decodeString, which takes a decoder and input. It returns an Elm
result, which will be Ok if we were successful, or Err if we had an issue.
Normally, if you’re decoding HTTP responses and so on, you won’t ever call this
function manually, the library you’re using will do it for you. It is really
useful for testing decoders though!
Note: if you’re more familiar with Elm decoding you might be aware of some
extra Elm packages that exist to make JSON decoding easier. We’ll cover those in
a future tutorial; for now I’m sticking to the core Elm library only.
I can run my decoder like so:
Decode.decodeString userDecoder """{"name": "Jack"}"""

By wrapping the JSON input with three quotes on each side, I avoid having to
escape the quotes in the JSON (three quotes is a multiline string in Elm where
you can use double quotes without escaping them). This gives us back:
Ok { name = "Jack" }

Which is perfect, and exactly what we want!
Type aliasing
It’s pretty dull to have to repeat the type { name : String } throughout this
imaginary example, so I can instead type alias it:
type alias User = { name : String }

When you define a type alias in Elm, you not only get the alias but User is a
constructor function:
User : String -> User

This means that I can call:
User "jack"

And get back:
{ name = "Jack" }

We can use this to our advantage. Recall that our userDecoder looks like so:
userDecoder : Decode.Decoder { name : String }
userDecoder =
Decode.map (\name -> { name = name })
(Decode.at [ "name" ] Decode.string)

Firstly, we can change the type annotation:
userDecoder : Decode.Decoder User
userDecoder =
Decode.map (\name -> { name = name })
(Decode.at [ "name" ] Decode.string)

And then we can update the function that creates our User:
userDecoder : Decode.Decoder User
userDecoder =
Decode.map (\name -> User name)
(Decode.at [ "name" ] Decode.string)

But whenever you have something of the form:
(\name -> User name)

Or, more generically:
(\x -> y x)

We can replace that by just passing the function we’re calling directly, leaving
us with the decoder:
userDecoder : Decode.Decoder User
userDecoder =
Decode.map User (Decode.at [ "name" ] Decode.string)

This is the most common pattern you’ll see when dealing with decoding in Elm.
The first argument to an object decoder is nearly always a constructor for a
type alias. Just remember, it’s a function that takes all the decoded values and
turns them into the thing we want to end up with.
An alternative to Decode.at
The decoding library also provides Decode.field, which reads out the value in
a particular field.
Decode.field "foo" Decode.string is the equivalent of
Decode.at ["foo"] Decode.string, but some find it reads a bit nicer.
Decode.at has the advantage of accepting a list to access nested fields, but
if you don’t need that you could use Decode.field.
— these two decoders are equivalent

userDecoder : Decode.Decoder User
userDecoder =
Decode.map User (Decode.at [ "name" ] Decode.string)

userDecoder : Decode.Decoder User
userDecoder =
Decode.map User (Decode.field "name" Decode.string)

Decoding a more complex JSON structure
Now we’re a bit more familiar with decoders, let’s look at our API and dealing
with the data it gives us.
The User Type
Our application is dealing with a User type that looks like so:
type alias User =
{ name : String
, age : Int
, description : Maybe String
, languages : List String
, playsFootball : Bool
}

The only piece of data a user might be missing is description, which is why
it’s modelled as a Maybe String.
The Data
Keeping in mind the above type we’ve got, here’s the API response we’re working
with:
{
"users": [
{
"name": "Jack",
"age": 24,
"description": "A person who writes Elm",
"languages": ["elm", "javascript"],
"sports": {
"football": true
}
},
{
"name": "Bob",
"age": 25,
"languages": ["ruby", "scala"],
"sports": {}
},
{
"name": "Alice",
"age": 23,
"description": "Alice sends secrets to Bob",
"languages": ["C", "scala", "elm"],
"sports": {
"football": false
}
}
]
}

Immediately you should notice some important features of this response:

All the data is nested under the users key
Not every user has a description field.
Every user has a sports object, but it doesn’t always have the football
key.

Granted, this example is a little extreme, but it’s not that common to see APIs
that have data like this. The good news is that if you have a nice, friendly,
consistent API, then this blog post will hopefully still help, and you’ll have
less work!
When dealing with data like this, I like to start with the simplest piece of the
puzzle and work up to the most complicated. Looking at the data we have, most of
the fields are always present, and always of the same type, so let’s start with
that and ignore the rest of the fields.
Let’s create the userDecoder that can decode a user object. We know we have
five fields, so we can use Decode.map5 to do that. The first argument we’ll
give it is the User type, which will be the function that constructs a user
for us. We can easily decode the name field, which is always a string:
userDecoder =
Decode.map5
User
(Decode.at [ "name" ] Decode.string)
— more fields to come here

And we can do the same for age, which is an integer:
userDecoder =
Decode.map5
User
(Decode.at [ "name" ] Decode.string)
(Decode.at [ "age" ] Decode.int)
— other fields to come, hold tight!

And we can do the same for languages. languages is a list of strings, and we
can decode that by using the Decode.list decoder, which takes another decoder
which it will use for each individual item. So Decode.list Decode.string
creates a decoder that can decode a list of strings:
userDecoder =
Decode.map5
User
(Decode.at [ "name" ] Decode.string)
(Decode.at [ "age" ] Decode.int)
— we’ll decode the description field here in a mo
(Decode.at [ "languages" ] (Decode.list Decode.string))
— we’ll decode the sports object here in a mo

A top tip when you want to test decoders incrementally is that you can use
Decode.succeed to have a decoder pay no attention to the actual JSON and just
succeed with the given value. So to finish our decoder we can simply fill in our
missing fields with Decode.succeed:
userDecoder =
Decode.map5
User
(Decode.at [ "name" ] Decode.string)
(Decode.at [ "age" ] Decode.int)
(Decode.succeed Nothing)
(Decode.at [ "languages" ] (Decode.list Decode.string))
(Decode.succeed False)

That makes our decoded description value always Nothing (recall that
description is a Maybe), and our playsFootball value always False.
Order of decoders
Something that I failed to realise early on when I was getting used to JSON
decoding is why the decoders above are ordered as such. It’s because they match
the ordering of values in the User type alias.
Because the User fields are defined in this order:
type alias User =
{ name : String
, age : Int
, description : Maybe String
, languages : List String
, playsFootball : Bool
}

We have to decode in that order, too.
Decoding maybe values
If we have a key that is not always present, we can decode that with
Decode.maybe. This takes another decoder, and if that decoder fails because
the key it’s looking for isn’t present, it will be decoded to Nothing. Else,
it will be decoded to Just val, where val is the value that was decoded.
What this means in practice is that to decode a maybe you simply write the
decoder you would write if the field was always present, in our case:
(Decode.at [ "description" ] Decode.string)

And we then wrap it in Decode.maybe:
(Decode.maybe (Decode.at [ "description" ] Decode.string))

And that’s it! We’re now nearly done with our decoder:
userDecoder : Decode.Decoder User
userDecoder =
Decode.map5
User
(Decode.at [ "name" ] Decode.string)
(Decode.at [ "age" ] Decode.int)
(Decode.maybe (Decode.at [ "description" ] Decode.string))
(Decode.at [ "languages" ] (Decode.list Decode.string))
(Decode.succeed False) — just this one to go!

Decode.map
It’s time to get a bit more complex and decode the sports object. Remember that
we just want to pull out the football field, if it’s present, but set it to
False if it’s not present.
The sports key will be one of three values:

{}
{ "football": true }
{ "football": false }

And we use it to set the playsFootball boolean to True or False. In the
case where the football key isn’t set, we want to default it to False.
Before dealing with the case where it’s missing, let’s pretend it’s always
present, and see how we would decode that. We’d create a decoder that pulls out
the football field, and decodes it as a boolean:
Decode.at [ "sports", "football" ] Decode.bool

That would pull out the football key in the sports object, and decode it as
a boolean. However, we need to deal with the football key being missing. The
first thing I’m going to do is define another decoder, sportsDecoder, which
will take the sports object and decode it:
Decode.at [ "sports" ] sportsDecoder

sportsDecoder =
Decode.at [ "football" ] Decode.bool

This is equivalent to the previous example but we’ve now split the code up a
little. Remember earlier that we used Decode.succeed to make a JSON decoder
succeed with a given value? That’s what we need to use here. We effectively want
to try to decode it first, but if it goes wrong, just return False. If we were
writing our decoder out in English, we’d say:

Try to find the value in the football field and decode it as boolean.
If something goes wrong, don’t worry about it, just set the value to False.

It turns out that Elm gives us Decode.oneOf, which does exactly that!
Decode.oneOf takes a list of decoders and will try each of them in turn. If
anything goes wrong it will try the next decoder in the list. Only if none of
the decoders work will it fail.
So the first thing we can do is wrap our existing sportsDecoder in a
Decode.oneOf call:
sportsDecoder =
(Decode.oneOf
[ Decode.at [ "football" ] Decode.bool ]
)

That will work when the field is present, but now we need to cover the other
case and always return False:
sportsDecoder =
(Decode.oneOf
[ Decode.at [ "football" ] Decode.bool
, Decode.succeed False
]
)

With that change, we decode the value if it exists, or we set it to False.
We’re done!
Conclusion
I hope this article has gone some way to showing that Elm’s decoding isn’t quite
as scary as it first seems. Yes, it’s not always immediately intuitive, and
takes time to get used to, but once you get the hang of it I think you’ll find
it really nice to be able to so explicitly deal with JSON and decode it into
your application’s types.
If you’d like to look at the code, I’ve
got a small app on Github
that uses the decoders in this article, and you can
find me on Twitter (or the Elm slack
channel!) if you have any questions.

Link: http://javascriptplayground.com/json-decoding-in-elm/