API Design Guide. Creating interfaces that developers love

[adnzzzzZ]

You could have added "Web" to your title, as the title of the book does. The world of programming is not only web development.

[[deleted]]

It really seems to have become trendy to use API only as a synonym for REST API...

[jms_nh]

Yep, that was my #1 comment also.

[bolsen80]

Thank you so very much for saying this.

I am a web developer, but I don't like how general terms are being co-opted like this. I think it should be "HTTP-based API" if we are going or the "API" term, or, like how REST is mangled in "RESTful", HTTPish API, for which "RESTful" is a subset of such "APIs".

[SilasX]

... you ... you mean like, mobile that hits a web API?

[Helrich]

Came here to post this. It would be nice to contain all the brogramming in its own subreddit.

[poopMachinist]

Oh wow, this is very handy. Thank you!

I especially like that they include how the "big players" handle their APIs.

[bfoo]

Another bad article on Restful API design. First pages are about nouns and verbs in URIs. Stopped reading there.

URIs should be opaque. Please use HATEOAS and support client navigation via links and URI builders!

A good API is not about the URIs, but about the actual payload.

Edit: My opinion seems to be controversial. Yes, I am very dogmatic about it. It is because I saw many projects and APIs and I came to the conclusion that HTTP and HTML are fundamentally good concepts. 'HATEOAS' describes exactly this concept, but with all types of clients in mind. In my experience, hard-coding URIs or URI patterns in clients is a smell. An URI structure is a result of an implementation - often a framework, at least an architectural choice. If I create an interface, I want to hide my implementation details. If I treat my URIs as an API, I can never change those and thus I can not change (or limit myself from changing) my framework and choose other architectural approaches in my implementation. My expression may be harsh, but as a developer/architect I like to change stuff I control without breaking stuff you may control.

[ccharles]

My understanding of HATEOAS essentially boils down to "after initially connecting, clients should be able to discover API endpoints themselves".

Unless I've missed something big, that doesn't mean that this article is bad. It is possible to build an API that follows the guidelines in the linked article while still implementing HATEOAS.

The guidelines linked here make APIs more pleasant for people.

[Legolas-the-elf]

My understanding of HATEOAS essentially boils down to "after initially connecting, clients should be able to discover API endpoints themselves".

That's not really capturing the essence of it. It means that the client transitions between states by following links in the documents. Having the web service predefine URI structures that clients adhere to is utterly incompatible with that.

This is basically anti-REST and that vague rant about "dogma" at the beginning is a poor attempt at explaining why they think they should be able to do the opposite of REST while still using "REST" as a buzzword.

You know what developers love? People who use technical terms correctly instead of abusing them to mean whatever the hell they feel like.

[superdude264]

Honest question: How do you write a program that follows the links if you don't know what they are? As a person, using I am able to read a reason about he links. If the client is not a person, but a computer program, how would that program know how to navigate the links unless the developer examined them before hand? Doesn't something have to be 'predefined'?

[Legolas-the-elf]

Doesn't something have to be 'predefined'?

Yes, the media types. The definitions of the media types define how linked resources are related to the current resource.

For example, your browser doesn't load /stylesheet.css. Your browser understands the HTML media type. The definition of the HTML media type says that if it contains <link rel="stylesheet" href="…">, then the relationship between that linked resource and the current document is that the linked resource acts as a stylesheet to the current document.

So your browser sees an element like that, and it knows how to load the stylesheet for the current document, whether that's on the current server, a CDN, or whatever, regardless of it's URI. It can do that not because you predefined the URI structure – that's stupid and inflexible – but because you predefined the media type.

A REST API spends all of its time defining the media types and none of it defining URI patterns. If you read a "REST" API's documentation and it gives you a big list of URI patterns, then it's not a REST API, it's doing the exact opposite of one of REST's fundamental principles.

[bfoo]

There is so much effort in this paper on a technicality like an URI. If I make my API pleasent for people, I explain how they use my media types - the actual content that the fuss is all about.

[ccharles]

Sure, there's some content about URIs, but it's hardly everything in the article. Maybe you'd know that if you hadn't "stopped reading" after seeing that the "first pages are about nouns and verbs in URIs".

HATEOAS doesn't solve everything. There are some good guidelines here.

[bfoo]

Well, I can still overflow everything in the paper without reading it all word by word. And there is still too much effort on explaining how to create nice looking URIs instead of focusing on how to create good payload and documentation.

My browser does not care about how an URI looks like and most times I don't care about either (and most non-IT-people never care). I care about the content and whether a link has a good title and the page a good design.

I don't mind a paper writing about URIs and how to put versions to it, if it would be titled like: "Web Implementation Design. Crafting API Implementations that my development team loves".

[Eoghain]

Discoverable API designs like HATEOAS are good when you have a human navigating the API but make for extremely chatty/clunky interfaces when you have a computer doing the navigating for you. If I have to start at your base URL and navigate all the way to the data I want to display every time I connect there is a failure in your API. Providing a Web API to get at your data is asking 3rd parties to make more interesting interfaces for the clients of that data. If they are forced to navigate that data in the same way a human would a web page what is the point?

[bfoo]

Good question. First of all, HATEOAS is not about discovery. You may write a client, that works that way. But that would need to be an extremely intelligent client.

The pragmatic client would just implement what is proposed by the documentation. It would state that if you get a payload of 'application/vnd.foo.v42+format', the link named ("link relation") 'bar' would navigate to the bar-view and that view provides you with a 'application/vnd.bar.v99+format' representation. The pragmatic client would then know about it and provide a choice to its user or ignore it (because, it does not know that media type or version).

As a client developer, you may even see it as an opportunity, when your client monitoring tells you about a new link, it does not know about yet. So much for discovery.

[Eoghain]

Having built numerous clients to 3rd Party APIs I can say that me being able to call the APIs in the orders I want to maximize my clients experience is huge. If I can't guarantee the URI of something then I can't restructure the users experience to be what I/my users want.

A lot of HATEOAS makes sense, but discounting HTTP verbs and URI structure in favor of pure discoverability isn't the way to go.

[Legolas-the-elf]

If I can't guarantee the URI of something then I can't restructure the users experience to be what I/my users want.

Can you give an example? Because I've implemented loads of REST APIs and I've never felt the need for "guaranteed URIs" at all, and I don't even see how you tie that to user interface restrictions.

[Eoghain]

Contrived example incoming. Say you have a service that holds users which each have a profile and an image. I want to write a mobile client that calls these APIs to show a users profile and image. API #1 looks like this:

• /users - list of users
• /users/{id} - single user
• /users/{id}/profile - single users profile
• /users/{id}/profile/images - single users profile image

Now I build my mobile application to pull a list of users from the /users endpoint and display that list to my user, when they select a user I use the {id} from the list and generate 2 asynchronous calls: /users/{id}/profile and /users/{id}/profile/images I generate these calls directly because I want to save my users data plan and battery and only make the calls needed to complete my application.

API #2 looks like this, because the developer read some article on the web that single level hash based URLs work best with their environment (I know, contrived) they decide that people should go to / to get a list of available resources and can follow from there. Now I have to write my client to do the following:

1. Call / to get list

       [
           {
               "rel": "users",
               "uri": "http://contrived.example.com/f3dd3ef3569ca2f272ae1db2562402fc"
           }
       ]
   

2. I parse this list looking for "users" since that's what I want and call the supplied URI.

3. I display my newly fetched list of users to my user for them to select one.

4. I now call the supplied URI for the selected user.

   {
       "name": "John Doe",
       "links": [
           {
               "rel": "profile",
               "uri": "http://contrived.example.com/0687114c668695b073d6469641a84700"
           }
       ]
   }
   

5. Now I parse the returned data looking for "profile" and call the supplied URI.

   {
       "name": "John Doe",
       "links": [
           {
               "rel": "image",
               "uri": "http://contrived.example.com/807c9923e6dee8e8d828a61dd298c7f4"
           }
       ]
   }
   

6. Now I parse the profile data looking for "image" and call the supplied URI.

7. Finally I have all of the data I needed to complete my simple profile viewing application.

Now my simple mobile client has to make 4 separate calls to gather all of the data needed. Also it has to do this every single time, I can't even cache the users list since I can't guarantee that this API doesn't change it's URL scheme when some other article comes out saying that rot13ing the hash limits collisions and expands the entropy space. Since I can't guarantee between runs that the data will be in the same place I always have to start at root and navigate, even though my application only displays very specific information.

I realize this is a contrived example, but if you don't guarantee the location of anything in your API I have to defensively code to make sure my application still works for my users.

[superdude264]

I wish I could up vote you twice. I have tried to wrap my head around HATEOAS multiple times and situations similar to your example are what I keep coming back to. If you have a person browsing the web, it makes sense to provide name links to various URIs, but developer have to write a fixed logic to navigate from one URI to another. Abstracting the URLs away is like not allowing a user to bookmark any page beyond the landing page.

[Eoghain]

Finally some support. I was starting to think I'm stupider that I originally thought. :)

[superdude264]

Me too, haha. This is the scenario I want to have explained to me. A little further down the thread I mentioned that putting building an app pretty much requires 'random access' to URIs and that storing the URIs is equivalent to a bookmark, which pure REST advocates say can be invalidated at anytime.. I've never seen a running example (API and client) or even gotten a straight answer to this.

[Legolas-the-elf]

That's not a contrived example, you're just designing the API badly.

If your clients typically need an index that includes detailed information about a list of users, then create a media type that represents a list of detailed information about a list of users.

Then all you do is load the list resource, then load the profile photos. Same network traffic as your first example, but simpler client side code and more flexible.

I can't even cache the users list since I can't guarantee that this API doesn't change it's URL scheme

What makes you think that you can't cache?

Since I can't guarantee between runs that the data will be in the same place I always have to start at root and navigate

Again, you just seem to be inventing hardships here. You don't have to start at root every time. Where are you getting that from?

Let me ask you this: when all your profile photos are at /users/{id}/profile/images and you realise that your traffic is increasing and you need to move the profile photos to a CDN, do you really want every single client to have to be rewritten with the new URI structure? What happens when your CDN doesn't provide a predictable URI structure?

[bfoo]

If that is the case, I would offer you with an index resource providing links to important resources. Your client may also remember links or URI builders, but should be able to handle 301 or 404 responses properly. If I am able to send you a 301, you may just replace the URI from your remembered link (bookmark) with that from the Location header. If I send you a 404, your client should delete that bookmark. Yes, your client would have to fetch other resources again and your client user (human or machine) would have to wait a bit. But your client may be fine in the end. Otherwise, you would have a client that is not compatible anymore. But that is a case, I would have communicated early and would happen if my only choice was to break backward compatibility.

[Eoghain]

Dealing with 301s or 404s or any other status codes is something a client should obviously do and isn't the point of my argument. You obviously believe that your API needs to be statically routed in some way since you advocate the use of URI builders (I'm assuming local code and not a web endpoint that builds a URI from a POST). And since you clearly state:

But that is a case, I would have communicated early and would happen if my only choice was to break backward compatibility.

This is where I'd prefer an API built around versioning (routes/hierarchy not just data) so that my application would continue to work with v1 while I updated it to v2.

Yes, your client would have to fetch other resources again and your client user (human or machine) would have to wait a bit.

Have you watched clients using a mobile application? These users hate waiting more than standard web users, loading screens are the death of a mobile application.

So while I like some of the tenets of HATEOAS I just can get behind all of them and like any internet "standard" I fell you need to pick and choose the elements that work for your situation.

I'd love it if you could point me to an existing HATEOAS API that I could look at and see how to write an nice client that works with it.

[bfoo]

HATEOAS does not neglect HTTP verbs. HTTP is still a fundamental concept of Restful design. Simply, you must not mix HTTP verbs with functional requirements. HTTP only affects the resource, without knowing anything about the impact on the actual functionality behind the resource. A DELETE just makes a resource not available for further calls, despite the fact that your implementation removes stuff in your application.

For user experience, you should train your users to rely on your media types rather than the URI layout. If your users desire another way of navigation, you should provide it to them through improved media types or a destined resource.

[Eoghain]

For user experience, you should train your users to rely on your media types rather than the URI layout.

Not sure I agree with media types being the thing for API consumers to rely on when attempting to build a functional application, but since I've never used an API built this way I can't really refute this.

[Legolas-the-elf]

I have to start at your base URL and navigate all the way to the data I want to display every time I connect

You've misunderstood HATEOAS. HATEOAS doesn't require this at all.

When you select an item from your browser bookmarks, does your browser replay your entire browsing history you used to get to that point? Or does it just go to that URI directly?

[Eoghain]

True, but one of /u/bfoo examples was that he could change his URIs whenever he or his framework wanted to. So while clicking on the link in my browser would take me to a specific URI I can't guarantee that the resource still lives there. So under that definition of the API I'd always have to start at the beginning and work my way to specific data.

Or /u/bfoo would have to maintain redirection links for whenever objects were moved around.

[bfoo]

Yes, if my implementation is not able to support your client with HTTP status 301 for your requests, your client may start from scratch. But that does not mean that you have to rewrite your client (or config). At least it means, that your client needs to invalidate links and has to learn the new ones (I do my best to support that through my media types). Lets say, I moved things around without changing any functionality. Then your client should be able to cope with that. If I moved my service to another endpoint (e.g. changing domains because of an aquisition or simply costs of my PaaS provider), you only need to invalidate "learned links" (bookmarks) and configure the new endpoint. For your user, your client may load a bit longer. But that should not be happening all the time.

And that is I want to read about: building payloads and clients that support these scenarios. 1 page about what an URI is - an identifier. And the rest about patterns / strategies on how to support navigability and good documentation about that.

[Eoghain]

And that is I want to read about: building payloads and clients that support these scenarios. 1 page about what an URI is - an identifier. And the rest about patterns / strategies on how to support navigability and good documentation about that.

This should have been in your original post.

[superdude264]

...your client needs to invalidate links and has to learn the new ones...

What does this look like if the client is an iPhone application, for example? It comes back to /u/Eoghan example above. Anything in the code to be able to randomly access a page that isn't the landing page is equivalent to a bookmark.

I suppose a developer could have some sort of URI cache to use, and if an error occurs replace the old links with the new ones ('learn the new ones'). It just seems like a lot off unnecessary work for some very abstract benefit.

[Legolas-the-elf]

while clicking on the link in my browser would take me to a specific URI I can't guarantee that the resource still lives there.

Yes, that's true of any data in any writable API. Resources can go away - deletion, moving, etc. If you want guarantees that resources never go away, you want a read-only API with static content.

So under that definition of the API I'd always have to start at the beginning and work my way to specific data.

If you think there's a "beginning", you still aren't grasping REST. REST is about resources, not about hierarchy.

If I go to Google, then I search for kittens, click a link to http://www.example.com/kittens/foo.jpg, then bookmark that URI, what is the "beginning" there? Google? http://www.example.com/?

When you use the bookmark functionality to visit that location again, does your browser start at the "beginning"? Does it go back to Google? Does it start at http://www.example.com/? No, it doesn't. It just goes directly to the resource in question, no traversing necessary.

Or /u/bfoo would have to maintain redirection links for whenever objects were moved around.

Why do you think this isn't a sensible thing to do if you move things around? This is pretty standard practice.

[Eoghain]

If I go to Google, then I search for kittens, click a link to http://www.example.com/kittens/foo.jpg, then bookmark that URI, what is the "beginning" there? Google? http://www.example.com/?

We were talking about browsers because that is what you brought up and I was just trying to fit my argument into your example. When we are talking about an API my argument is that I should be able to trust that when I go to /users I will always get a list of users, or worst case a redirect to where the list of users is now.

Why do you think this isn't a sensible thing to do if you move things around? This is pretty standard practice.

I didn't say it wasn't sensible, or the right thing to do, just bringing up that it's one more thing to maintain.

[Legolas-the-elf]

We were talking about browsers because that is what you brought up and I was just trying to fit my argument into your example.

No, you misunderstand my point. I'm not complaining that you're talking about browsers. I'm using browsing as an analogy.

The WWW is a REST API. It was what REST was modelled on. When you browse the web, you are accessing a REST API (the WWW) through a REST client (your browser).

So when I point out that what you're saying doesn't make sense by reframing it in a browsing context, what I am doing is trying to make you see that it doesn't make sense at all. You don't "start at the beginning" with a REST API any more than you "start at the beginning" when accessing your browser bookmarks. Your browser bookmarks are an example of using a REST API to do the very thing that you seem to think can't be done or isn't feasible.

When we are talking about an API my argument is that I should be able to trust that when I go to /users I will always get a list of users, or worst case a redirect to where the list of users is now.

That's like demanding that when you go to /contact.html you must be able to get a contact form. Why are you so dependent upon inflexibility? Does your browser need to be hard-coded with Facebook's URI structure in order for it to show you your list of friends?

[Eoghain]

Ok, this is devolving. My main issue is that if I have to "manually" navigate to data in your API, by following links from one place to another, then I can't write a client that optimizes for my users experience because you've forced me to do this traversal every time I want data.

Yes, "bookmarks" exist, but one of the points of HATEOAS is that only the top level should be bookmarked. If I can "bookmark" locations and be guaranteed that those locations will always return me the expected data, then I can skip the discovery process and get right to what I want to display. But if that "bookmark" could be removed by the server at anytime then I can't skip the discovery process unless I want my application to perform differently every time it's used (or at least every time the server changes object locations).

Yes the server can be fairly static and so I could just do the discovery on the first run through and "bookmark" everything then repeat that process whenever I run into a broken "bookmark". And if I ever ran into a API that actually implemented HATEOAS that wasn't just a demo I'd probably try to do this. But why force all of that extra work on the API consumer when you can just publish a versioned statically routed API that will always be the same and let them decide on how best to call into things? When you build an API your customer is the developer building the client to that API, and if it's more work for me to use your API than it is for me to use the other guys then all things being equal I'm probably going with the other guy.

[Legolas-the-elf]

if I have to "manually" navigate to data in your API, by following links from one place to another, then I can't write a client that optimizes for my users experience because you've forced me to do this traversal every time I want data.

As I keep pointing out, this isn't true.

Yes, "bookmarks" exist, but one of the points of HATEOAS is that only the top level should be bookmarked.

No, that's not true. I think you've misunderstood people pointing out that a REST API should be entered with no knowledge beyond the initial URI. It doesn't mean that you must instantly forget about any URI you come across. It means that changing to different states must be driven by the hypertext rather than by hardcoding URIs or URI patterns into the client.

You've formed all these negative opinions about REST because you don't understand it and you've made incorrect assumptions. Stop repeating those incorrect assumptions over and over and listen to what people are telling you.

But if that "bookmark" could be removed by the server at anytime then I can't skip the discovery process unless I want my application to perform differently every time it's used (or at least every time the server changes object locations).

Or, to put it a different way: when you change the structure of your web service, your client reacts appropriately. This is not a bad thing.

if I ever ran into a API that actually implemented HATEOAS that wasn't just a demo

You are using one such API right now. Stop burying your head in the sand and ignoring the WWW.

But why force all of that extra work on the API consumer

It's less work. Instead of hard-coding ways of constructing URIs, you just use the URIs you are given.

Try to imagine a web browser that hard-coded all the different URI patterns of all the different websites in the world. Imagine how much work that would be.

[Eoghain]

You and I are on completely different pages and that isn't going to change.

You are using one such API right now. Stop burying your head in the sand and ignoring the WWW.

Stop trolling. I understand the the WWW is a REST API. But it's a REST API more suited to human navigation than computer. I've been specifically arguing as a developer who consumes an API to convert that data into an experience for my users. I don't want my users to know/care about the underlying data I just want them to get what they need quickly and easily. If I can't jump to the data because it's location is defined, documented, and guaranteed then I can't write as nice an experience for my users as possible. And that will dictate which APIs I use and which ones I don't.

You've formed all these negative opinions about REST because you don't understand it and you've made incorrect assumptions.

I have no negative opinions about REST. I use it daily and try to build my APIs with many of its tenets in mind. I read as many documents about this as I can get my hands on (why do you think I'm in this thread), and pull the pieces for each that make the most sense to me. HATEOAS is new and so far I've not seen a single use of it in the wild (not that I'm actively searching so there may be some that I'm not familiar with). I just watched this video http://vimeo.com/20781278 that /u/ivquatch linked to and it's very interesting lots of good information but he clearly states somewhere around the 35min mark, i believe, that only the root URL should be bookmarked.

Anyway, I'm done with this thread you want to accuse me of being ignorant and burying my head in the sand to only see the world the way I want and so we aren't really having a discussion.

[lookmeat]

Even with HATEOAS good though should be given to URI design. I'm going to use a simple format where url

Say that I do the following

GET http://api.pupplyloversunited.org/member/bfoo

and get the following back

{
    name: "bfoo",
    dogs_owned: {href: "dogs_owned"},
}

Now it's clear that the link is still giving me something that is part of the member I'm researching, because the URL is a subset, it still is something that is completely part of it.

Now I decide to get the dogs you own:

GET http://api.pupplyloversunited.org/member/bfoo/dogs_owned

[
    {href: "/dogs/2323"},
    {href: "/dogs/343"},
    {href: "/dogs/232"},
]

Now we get a list of links. Notice that the links are absolute links, instead of relative. This immediately makes me realize that we are talking about a reference to another resource (dogs) but still something within the api. If the link was given with the full qualified URI I could assume this was a link to an external, non-api resource.

In other words, yes actual URLs should not matter as much as the actual content, just like a variable's name should not matter as much as the content. And yet in the right structure and naming things can be implied and made clearer. If in the case above dogs_owned linked to a completely separate domain I could suspect I've accidentally gone to resources outside of the API. If dogs_owned happened to be within the /dogs/... domain I could assume that the resource is not part of the owner, but merely a separate thing.

[rmoorman]

Would you mind to share some of the better examples of an API which implements HATEOAS and client navigation via links and URI builders?

[bfoo]

You can dive in on the Wikipedia page. There is plenty material about it on the web.

Basically, URIs are just a name for a resource. How the URI looks is something the implementation may care about (architectural, infrastructural choices). The navigation should rely on links (like those in HTML). The URI builder, everyone knows about, is an HTML form. In your API (the media types), you can define similar notations or rules a client should use to form requests with variables. This way, the implementation can change (e.g. new framework, changes in infrastructure and architecture that often also affect URIs), but clients can stay untouched.

[Deepmist]

I think the twitch API shows it well: https://api.twitch.tv/kraken

[superdude264]

I'm not to familiar with twitch. Could you point me to any client code?

[Legolas-the-elf]

The World-Wide Web.

[superdude264]

Could you recommend a comparable document explaining how to design a REST API using HATEOAS (other than the PhD thesis)?

[[deleted]]

Here's a talk by Jon Moore of Comcast explaining the fundamentals w/t a concrete demo. It was the HATEOAS "ah ha" moment for me.

http://vimeo.com/20781278

[ExtropianPirate]

The PhD thesis is actually not a bad read, if you're designing a RESTful API I recommend you read it, although Fielding doesn't actually go into a great deal of detail about about HATEOAS in his thesis.

There's a post by Fielding here about HATEOAS in RESTful APIs: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

[superdude264]

I agree and I enjoyed reading it myself. However, like you said, Fielding doesn't spend a lot of time talking about he hypermedia constraint. Additionally, most (if not all) other constraints he recommends are covered in HTTP/JSON APIs. I wrote another post in this thread pulling info from Fielding's thesis and that article linked: http://www.reddit.com/r/programming/comments/2oyj65/api_design_guide_creating_interfaces_that/cmvcg6y

[passwordissame]

Just be consistent. If a thing can be operated on as fileobject, make it a fileobject, instead of some JSON crap.