Why Is My Web Service API Crappy?
I knew that I hard written a crappy web service API but why was it so crappy? Was it because, no matter what, it only returned JSON data? Was it because the arguments were passed via their positions in the URL? Was it because the version of the API was part of the URL? Was it all of the above and, if so, how was it supposed to be done?
I’ve been doing a lot of research and reading and I think I have a better idea of how this all should work. For the sake of getting all of my ideas down in one place, I’m going to crudely regurgitate my current thinking in this article.
Table of Contents
Why It’s Crappy
Right off the bat, I knew my API was weak in that it completely ignored the HTTP protocol. If you’re writing a service that speaks HTTP (and I was), you should probably know a little about how HTTP works (and I didn’t). This is the sort of thing those crazy REST people have been crowing about forever and now I can understand why: there’s a lot more meat in there than I had thought. For sure, it won’t solve all of your problems but it can solve some of them.
Another Way
If I made my API work a little bit more like how web servers already work, I could make things easier on everyone. I’ve always worked with a pretty lightweight framework to put these APIs together (i.e. CherryPy or Ring) and while this works okay, it left lot of the work of handling the various HTTP headers up to the me. I mostly skipped all of that stuff because I needed to get things done. If you’ve done the same, trust me, there’s no judgment here. But there are some good reasons to implement more of this stuff.
Give the Client What the Client Wants
For one thing, the client can tell you very specifically what kind of content it wants. Maybe it wants XML, if you really only serve JSON data you can go ahead and say so. Maybe the client does want JSON data but only an older version of your API. That’s right, you don’t need to put the version in the URL, finessing the requested content type can serve the same purpose.
Tell the Client Where To Go
I usually end up publishing more and more URLs as more functionality is needed. It all ends up looking very ad-hoc because it’s all very ad-hoc. But what if we provided pointers to the client instructing it where it should go to get the information that it wants or to change the state of a resource? We’d end up with much looser URLs and we’d be able to change them when we need to. While the client needs to do a little more work to parse out these URLs, I think it’s definitely worth it.
Provide Helpful Hints
The client can submit the “OPTIONS” request to find out what actions (i.e. “GET”, “POST”, “PUT”, etc.) are valid for the URL. Sure, you can put that in the documentation but the client can also discover it on their own. Much of the time the correct method will already be assumed; i.e. you POST to create a new resource and PUT to update an existing item.
URLs Can Represent State Changes
Once we get our API to the point where the client is following the links that we provided with our responses, we’re in a place where the state of a particular item changes as the URLs our API provides are followed. If a particular action isn’t valid for a particular item we simply do not provide a URL for that action.
This isn’t earth-shattering news, it’s just not something I hadn’t ever thought about before. But now that I’m thinking about it, it’s clearly a much better way to manage these state transitions.
Predictable Caching
Caching the results of your API can be a real pain point, especially if you’re not providing the information that the caching layer really needs to be effective. For the longest time I didn’t have to worry about putting a cache in front of my web service, they were all internal. Once I did, it was made clear to me how difficult my service was making it for everyone else.
A Really Simple Web Service
Okay, enough talk. What does an API that leverages these ideas actual look like? We’ll use a simple example, a web service that exposes a single to-do list. First, we want to get a list of all of the items on our list, this represents the main entry point to our API. I’m not going to list all the headers, just the relevant ones.
GET /todos HTTP/1.1
Host: somesite.com
Accept: application/vnd.com.somesite.todos-v1+xml
Date: Mon, 02 Apr 2012 01:04:15 GMT
Status: 200 OK
Content-Type: application/vnd.com.somesite.todos-v1+xml;charset=utf-8
ETag: "1f298572c3b90d4b4e9da5f25ea0ba9a"
Vary: Accept
<list xmlns="http://somesite.com/todo/schema/list"
xmlns:atom="http://www.w3c.org/2005/atom">
<todo>
<description>Write up demonstration of to-do API</description>
<atom:link rel="resource" href="http://somesite.com/todos/1023"/>
</todo>
<todo>
<description>Add more to-do items</description>
<atom:link rel="resource" href="http://somesite.com/todos/1312"/>
<todo>
</list>
In this example the client requests the current list of to-do items from the server and specifies that it accepts responses that are specifically version 1 of our XML API. Since the server knows what version of the API the client is using, it’s easy to provide the response in the format that the client really wants.
The server provides an “ETag” header with the response, in this case this is a value that will change every time the list changes. We could hash the list of to-do items or the unique id of each hash and it’s version; whatever makes the most sense. The client can then issue a conditional request to see if the resource has changed instead of simply re-fetching all the data.
Lastly, the server returns a list of the current to-do items and a pointer to where the client can find our schema for this list. We’re also leveraging the Atom Publishing Protocol to provide annotated links since that’s easier than writing up our own specification. Each to-do item has a link that indicates where that individual item lives, if the client needs to update or delete an item it knows where to go.
Next we create a new item by posting to the “todos” resource. Our API didn’t provide a link for this since it’s the common REST idiom.
POST /todos HTTP/1.1 Host: somesite.com Content-Type: application/vnd.com.somesite.todos-v1+xml <todo xmlns:atom="http://somesite.com/todo/schema/todo"> <description>Buy more coffee!</description> </todo>
The server creates the new item and then provides an affirmative response. It also sends a “Location” header, this indicates the URL for the new to-do item. The client can use that URL to fetch, update or delete the item. Lastly, the response body contains the complete content for the new to-do entry.
HTTP/1.1 201 Created Content-Type: application/vnd.com.somesite.todos-v1+xml Date: Mon, 02 Apr 2012 01:06:23 GMT ETag: "64d053d0bcf1b4de41f852509fd5ffc2" Location: http://somesite.com/todos/1314 <todo xmlns:atom="http://somesite.com/todo/schema/todo"> <description>Buy more coffee!</description> </todo>
With the location of the new resource in hand, the client can update this new to-do item at will.
PUT /todos/1314 HTTP/1.1 Host: somesite.com Content-Type: application/vnd.com.somesite.todos-v1+xml If-Match: "64d053d0bcf1b4de41f852509fd5ffc2" <todo xmlns:atom="http://somesite.com/todo/schema/todo"> <description>Buy more coffee! And more creamer.</description> </todo>
The client provides the entirety of the updated to-do item to that resource’s location (“PUT” always expects the entire resource). Note that the client provides the ETag of the resource in the “If-Match” header; if the to-do item has been updated by another client the ETag won’t match and the server will return a “412 Precondition Failed” instead of enacting the change.
The server then responds with a confirmation.
HTTP/1.1 200 OK Content-Type: application/vnd.com.somesite.todos-v1+xml Date: Mon, 02 Apr 2012 01:12:46 GMT ETag: "9cc7dc7e5f4a81e3d8daf752a622e84d" Location: http://somesite.com/todos/1314 <todo xmlns:atom="http://somesite.com/todo/schema/todo"> <description>Buy more coffee! And more creamer.</description> </todo>
And there you have it. The simple API above gracefully handles changes to the version of API if needed. Our URLs are provided with the relevant resource, hopefully cutting down on the number of URLs that end up hard-coded into the client. The client uses ETag data when updating resources to avoid conflicts. Our examples revolve around XML messages but there’s no reason this same service couldn’t also support JSON data.
Caching
In the example above, we didn’t do anything all that interesting to support caching of our API’s responses. Nonetheless, the list of to-do items and those items themselves could be cached somewhere between the client and the server; all “GET” requests are cache-able by default.
By adding headers to our responses we can exert some real control over how these responses are cached. We can specify a specific time, after which intermediate caches will need to fetch a fresh copy. We can specify what sort of cache is allowed to cache our data, indicating that the client can cache it locally but intermediate caches cannot. And, of course, we can tag certain responses as entirely uncache-able.
The client can save itself some transmission overhead by using conditional requests. In the example above, the client provided an “If-Match” header when updating a resource, ensuring that the resource was in the expected state. The client could use the very same logic when fetching the list of to-do items, it would only pull down the new list if it had changed.
Modeling a Protocol
We only covered some simple create, update and delete functionality in the example above. While this is a good fit for how most of us think about REST services it isn’t the whole story. This same method of including links with the resource can also be used to define a protocol, instructing the client on how a resource can be used. As the client invokes these URLs for a resource the state of the resource changes, the server instructs the client as to which changes are available by varying the supplied URLs.
There’s a great presentation out on slideshare by Dr. Jim Webber that walks through how such a protocol might work. His example is reasonably complicated, it works through ordering a coffee from coffee shop. He’s also written some interesting books on the subject.
You Don’t Need to Do it All On Your Own
There are some frameworks out there that make the construction of this kind of API much easier, Webmachine being the most popular example. These frameworks handle the lion’s share of the header parsing and exception handling for you, leaving you to implement only the functionality that makes your API work. In fact, I’m currently working on a library for Clojure that does just this (naturally, it’s a work in progress).
Comments powered by Disqus.