Suggested REST API Practices (2013)

In the two years since Backbone.js was released, thick-client JavaScript apps have dramatically changed how developers build APIs. New patterns such as Hypermedia, presenters, filtered parameters, single-endpoint records, and materialization-ready responses are refining what we think is valuable in an API.

Ember-Data is a data persistence library in JavaScript, one obviously built upon Ember.js. Many other libraries try to focus on minimal, lightweight solutions, but the Ember community is trying to create a complete solution for data persistence in the browser. After working closely with Ember-Data for a few months, I’ve begun to understand how well it implements many of the best practices for API creation, and where it still falls short.

From the my practical experiences with Ember and Backbone in thick-client JS apps, and considering where APIs and various libraries are heading, I believe there are several API strategies that can be considered best practices. I think these are best practices for any thick client app, whatever framework you build it in. And I’m hopeful that all of these patterns become more widespread in server-side and client-side libraries.

Create endpoints that focus on storage, retrieval, and permission. Endpoints that avoid understanding role, complex relationships changes, and states. They should do one thing well, and be side-effect free. APIs should facilitate the creation of client libraries that separate the concerns of serialization, API consumption, and domain modeling.

Here are five guidelines for constructing a REST API that stays sane as your app grows:

• Relate records via URLs. A quick word on Hypermedia APIs.
• First-class models. Treat most endpoints as first-class citizens, with root level URLs and models.
• Use presenters. Separate API concerns from your data layer.
• Complete responses. Endpoints should respond with all the data needed to materialize a model.
• Filter parameters. The robustness principle de-couples API and model concerns even more.
• Authorization via Tokens. Limit the use of sessions and cookies.

Relate records via URLs

Last year, Steve Klabnik coined the term Hypermedia API to describe APIs that more fully take advantage of features provided by HTTP. From my understanding, there are three important parts of a Hypermedia API:

• Records are related via URLs.
• All records are related.
• The root path provides an index of the API, much like websites providing navigation on their landing page.

Hypermedia APIs assume you have a canonical URL for each record, like database records each having a unique record identifier (table + id). By using URLs to relate records instead of ids, an API can remove an explicit tie to the server’s data models. A well constructed API client would no longer need to explicitly understand the URLs for it’s models. Instead, the API itself explains how domain objects map to certain URLs.

Consider this JSON response:

// GET /users/12
{
  "users": {
    "id": 12,
    "name": "Bobby",
    "_href": "//api.example.com/users/12",
    "best_friend_href": "//api.example.com/users/13"
  }
}

Instead of using _id to relate records via a foreign key, the language of http takes over. The client should materialize this response into a record, and without needing to understand the URL or type of the bestFriend, perform a query to fetch it.

It is important to note that Hypermedia APIs are a concept, not a protocol or standard of any kind. In one quick wrap-up on Hypermedia, Klabnik recommends you pick a specific implementation to build an API. The best options currently seem to be HAL and Collection+JSON. Creating a custom protocol is certainly possible.

Ember-Data does not support Hypermedia, though support for the idea has been voiced by core team folk. If you want to build an Ember app today, I suggest you use ActiveModel::Serializers and Ember-Data, but you should understand what Hypermedia APIs mean and prepare for using them in your application.

If you can find a good API adapter, or are willing to write one, Hypermedia will further distance your data models from the API representing them. It will lessen the code needed for the client. It will make much of my “First-class Models” suggestion moot.

But, with client library support so weak, it is hard to call Hypermedia a best practice today. I think it will become increasingly important, but the real-world hurdles to implementation are still significant.

First-class Models

Often, you find APIs dreamed up with nested routes like so:

/users/23/todo_lists/12/items

After building APIs that way, I’m now much happier with URLs like

/items?todo_list=12

There are two problems with nesting routes in an API:

They codify relationships you probably should not be confident of. In the previous example there is an assumption that items will always belong to a single todo_list. It is also assumed that todo_lists will belong to users. When you’re just starting out on an app and API, those are pretty bad assumptions.

The codification of relationships also leads to multiple models that only differ by relation. If there are two API endpoints for items:

/users/23/todo_lists/12/items
/users/23/items

Often the solution is to create two models. UserTodoListItem and UserItem. On the server, they are probably the same record. If the same item has been fetched from both endpoints, how do both model instances keep each other up to date?

You can work around this, but only if you use a complicated model store and adapter. And configure it for every endpoint. Which leads to…

They are harder to build a client-side adapter for. If I have a App.Item model, it should have a single resource dedicated to it. It can always know to save itself at /items, and never needs to understand that it could possibly be at /todo_list/32/items or /users/12/items. When fetching items with a certain scope, the scope is always a query parameter. URLs such as /items?todo_list_id=32 and /items?user_id=12 are simpler to support than nested URLs. A simpler adapter lets you focus on the relationships between objects in code, and not in your URLs.

Even if you are confident that users/23/todo_lists/12/items is the correct route, you have still added complexity to the client. It must understand not only potential query arguments to the route (?page=2), but must also understand that some values are to be embedded in the URL.

One URL for each type of object. Preferably a top-level path. Keep it simple.

Use Presenters

I see presenters and parameter filters as two sides of the same coin. Together, the two practices isolate server-side models from API concerns.

Presenters have some additional practical benefits:

• They give you an opportunity to compute additional properties you may want on the client.
• They provide an abstraction point for tests.
• They protect your API (and client code) from changes to the server-side data model.

@j3 has an excellent set of slides on how the presenter pattern came to use in Rails. Presenters are wrappers around models, responsible for decorating the model with new computed properties, and serializing relationships.

Example use of a presenter is be straightforward to follow, even if you don’t know any Ember.

Ember-Data and ActiveModel::Serializers are constructed to work well together. Defining a custom property in AM::S is simple:

class PersonSerializer < ActiveModel::Serializer
  attributes :first_name, :last_name, :full_name

  def full_name
    "#{object.first_name} #{object.last_name}"
  end
end

object in AM::S is the raw data model object. full_name, which is only needed on the client, does not need to be defined on the data model itself. This layer gives you an abstraction point to test the how data models are presented separately from the data-model tests themselves. This isolation allows the data models to be refactored (say, to a new storage backend) without changing the API tests.

Dan Gebhardt has a thorough set of Ember-Data/AM::S examples in his slides from Ember Camp, and I’ve provided some updates for the latest syntax changes in slides from an Ember NYC talk.

Complete Responses

The responses of API calls should be complete, allowing for materialization of models without context. Consider this API response:

// GET /users/12
{
  name: 'Bob Barker'
}

The API client will need to keep track of the type of object (User), and the id (12). The client is much simpler if all that information is included in the response:

// GET /users/12
{
  user: {
    id: 12,
    name: 'Bob Barker'
  }
}

With this response, the client can separate concerns over AJAX calls and materializing objects. It’s a simple change that Rails projects have already made the default, and allows Ember-Data to process responses like these:

// GET /users/12
{
  user: {
    id: 12,
    name: 'Bob Barker',
    todo_list_ids: [ 4, 5 ]
  }
  todo_lists: [
    { id: 4, name: 'shopping' },
    { id: 5, name: 'work' }
  ]
}

Note that each record is complete. The relationships are still passed with IDs, but additional data can be sidelined. Ember-Data just materializes all these into models, it isn’t concerned that only a user was expected.

Though Ember-Data uses keys to infer the record type, a property for type on each record would be just as effective. Inferring the type from a key avoids polluting the property keys of a record, but the important pattern is to include enough information in responses that the client does not need to guess the record type based on the URL or other context.

Filter Parameters

The robustness principle says “Be conservative in what you send, liberal in what you accept”. The quote comes from Internet pioneer Jon Postel talking about TCP in RFC 760.

Filtering parameters at the request level keeps API concerns away from data store models. It decouples client-side models from server-side models. Without this separation, models need to protect themselves from un-intended use via the API in all situations. Modifying an attribute from the console and modifying it via a 3rd-party initiated API request passes through the same protections.

Instead of protecting models from un-intended use, protect an API from unintended use. Explicitly whitelist request parameters you expect on the server, then be lazy with the model protections.

In Ember-Data and other client-side API consumers, filtering parameters on the server allows the adapter to be lazy about what may be submitted to the server. Pre-computed properties that have been added by a presenter can be easily ignored.

A simple example is a count. If a user has many lists, we may want to show how many lists they have without actually fetching all the lists to the client. Instead, let us presume the listCount is calculated during the API request for the user, and included in the response. When an Ember-Data model includes an attribute for the count:

App.User = DS.Model.extend({
  name: DS.attr('string')
  listCount: DS.attr('number')
});

Then the listCount would be sent back to the server when a user is saved. When the parameters are filtered, as in this Strong Parameters example:

class UsersController < ApplicationController

  def update
    @user = User.find(params[:id])
    # Here, the user is updated with cleaned parameters
    @user.update_attributes(user_params)
    expose @user
  end

  private

  def user_params
    # Of the parameters send to the server, we require
    # a key of user, then allow it to be a hash that includes name.
    # All other parameters are dropped, such as listCount.
    params.require(:user).permit(:name)
  end
end

Then the submitted listCount will be quietly disregarded by the server.

This use of server-computed properties and filters is not ideal, instead it is a temporary behavior we can take advantage of until Ember-Data supports read-only attributes. Other frameworks may support this behavior better.

The advantage of filtering parameters lies less in this use-case, and more in the separation of an API concern (request parameters) and a model concern (persistance).

Authorization via Tokens

If you’ve been building apps for a while, you’re used to the idea of a current_user. The authorization to access user-specific pages in your app is a long-term authorization. I think it’s best to move that authorization out of cookies and sessions, but that isn’t the kind of authorization I’m talking about.

Eventually, you will have times when you need to grant visitors access to resources only temporarily. A password reset page is one example, an invite to share access to something is another. In thick client apps, you may need to make several requests to the API for data to render a single page.

Say I invite a user to see a todo_list and collaborate on it. Our user visits /todo_lists/shopping/invite/dfe4. What requests do we need to make with temporary authorization?

GET /todo_lists/shopping <-- Display the list
GET /invite/dfe4         <-- Display the inviter's name
and eventually if they accept...
POST /users              <-- Create a user, add themselves to the list

On a traditional app, we could authorize the drawn page just based on the initial request. The eventual POST would need some temporary authorization. In a thick client app, we can bake this authorization into the data layer and a filter. That keeps it away from the logic about relationships between the list, the invitation, and the visitor.

Server-side, use some kind of filter for the authorization:

class TodoListController < ApplicationController
  attr_reader :authorization
  before_filter :require_current_user_or_token

  def show
    expose TodoList.find_if_permissible(params[:id], authorization)
  end

  private

  require_current_user_or_token
    if params[:token]
      @authorization = Invite.find_by_token(params[:token])
    else
      @authorization = current_user
    end
  end

end

Bonus points for creating an Authorization class of some kind to pass around into models.

On the client, set a token on the API adapter:

App.Adapter = DS.RESTAdapter.extend({
  buildURL: function(record, suffix) {
    var baseUrl = this._super(record, suffix);
    if (this.get('token')) {
      return baseUrl + "?token=" + this.get('token');
    } else {
      return baseUrl;
    }
  }
});
// Now you can App.set('store.adapter.token', '123'), then make normal requests
// using Ember-Data models.

This is Ember and Rails specific, but the pattern is to set authorization on the API adapter, and deal with it in filters on the server. This keeps API endpoints clean and maintainable, and allows you to consume the API via models without constant reference to the state of the visitor’s permissions.

Wrapping It Up

2013 is going to be a big, formative year for thick-client applications. By its end, I hope to see better read-only property support and Hypermedia support in Ember-Data. I hope to see more API clients encouraging better API design, and more server-side libraries that attempting to cover all the bases for a good API.

There are a few other best practices I skipped because they don’t deal with thick clients in particular. Versioning your API and using etags are two that come readily to mind. Despite the impressive array of successful thick-client JavaScript apps, I think we’re still in the early days of thick-client API design. I hope this map helps you make better decisions for your own apps. Decisions that keep your products flexible, and your code mangeable.

Many thanks to @dgeb for his feedback, to @fancyremarker for a thorough drubbing on the topic of Hypermedia APIs, and @bantic for a final read-through.