Webmachine - REST done right

Webmachine describes resources on top of HTTP to provide RESTful APIs. This is pretty much what every modern web framework states, right?

After writing some good amount of controllers on MVC frameworks you realise that it's complicated to keep a clean controller without using callbacks/middlewares to handle things like:

  • Authorisation;
  • Authentication;
  • Conditional redirection;
  • Caching;
  • Content representation.

On Rails code, for example, It's common to find before_filters (before_action on Rails 4) describing behaviour that should happen before handling the resource itself. Here's an example of discourse's ApplicationController:

These callbacks will run doing assorted things on pretty much every controller on the app. Just by looking on the name of the callbacks you can notice some redirection, authorisation, authentication and content representation logic. One important thing is that the order here matters and you will probably hit lots of issues if you change the order of the before_filters. There's also the problem that unit tests are complex as you need to test each callback indirectly.

Could it be less complicated? Of course!

HTTP Protocol

Webmachine describes the request handling as a state machine where HTTP status codes are end states. It implements the HTTP protocol as a collection of functions that will describe the flow into this state machine. Below you can see the initial state and some transitions:

Webmachine

Here's an example from webmachine-ruby:

This simple resource you will get automatically the following status:

  • 405 Method Not Allowed if it's not a GET;
  • 406 Not Acceptable if the request does not specify an Accept header that tolerates application/json;
  • 200 OK if everything is fine with and the correct content-type header will be provided.

Now let's handle when the order does not exist adding the resource_exists? method to the resource:

Now 404 Not Found will be given if resource_exists? returns falsy value.

Let's now add authorisation and authentication:

And again easily we will get:

  • 401 Unauthorized if is_authorized? returns falsy value;
  • 403 Forbidden if forbidden? returns falsy value

This is, in my opinion, a scalable way of describing your resources as the code will be as complex as the resources are. The framework already takes care of the nuances of the HTTP protocol keeping the code clean and extremely simple to unit test.

More

The original implementation of Webmachine is in Erlang but you can find webmachine in multiple programming languages: Erlang, Ruby, Clojure, Javascript, etc ...

There's also a neat presentation by Sean Cribbs: Resources, For Real This Time (with Webmachine)

Authentication on an Websocket API

Some months ago I was looking for a way to authenticate a websocket endpoint on Poxa's console. The answer in the end was easy. The same way that Pusher signs HTTP requests: a simple HMAC-SHA256 applied on the payload with a secret. The interesting part is that the payload is well defined. Here's an example from the Pusher website:

It's always composed as:

  • HTTP method;
  • URI path;
  • Query string composed of auth_key, auth_timestamp, auth_version and any other user parameter.
POST\n/apps/3/events\nauth_key=278d425bdf160c739803&auth_timestamp=1353088179&auth_version=1.0&body_md5=ec365a775a4cd0599faeb73354201b6f

This string is joined by \n and then signed using HMAC-SHA256 with the secret related to this key.

The last key value added to the query string is auth_signature which is the result of signing.

Which properties do you get with this?

  • Timestamp can avoid replay attacks in a certain period of time;
  • Auth version will let you handle changes on the authentication process and treat them differently;
  • The method won't let it be reused on the same URI with a different method;
  • The path won't let someone use the same method on a different URI;

In the end, any change on the request will mess up with the expected signature and invalidate it.

If you notice that a Websocket connection is started as a GET request, you can easily use this signing method to achieve the same properties.

On the client side, in this case the browser, I just used crypto-js with jQuery param:

On the backend side I used signaturex(written in Elixir), but you could use the original ruby library signature.

That's it! Don't forget to use SSL on top of it so you can encrypt everything around.

Unit testing your Cowboy Handlers

Cowboy is a "Small, fast, modular HTTP server written in Erlang.". This server uses handlers that describe the way your endpoints will behave during the lifetime of the request. Here's an example in Elixir from HTTParrot using cowboy_rest:

UserAgentHandler when initated upgrades the handler to a cowboy_rest (webmachine-like way of handling REST resources). After that it specifies that it will accept only GET requests and that replies only with JSON.

One interesting property that you get from functional programming is the ability of testing them without knowing any context. That's how I think you should test your cowboy handler (testing 2 callbacks only for the sake of simplicity):

In this case we are using meck to mock modules and expect on function calls.

The first test will guarantee that the handler will accept only GET requests. In this case, cowboy_rest will reply with status code 405 Method Not Allowed, but you don't need to test this behaviour as long as you respect the output expected for this callback. In this case it's a tuple of 3 items:

  • List of accepted methods as strings;
  • Request structure;
  • The state of the handler that can be used on next steps of the processing;

Notice that it's not important which values req and state have. The only important thing is that you are not changing them. The assertion expects the same value passed as req and state.

The second test is a little bit more complicated, but nothing special. First we expect that :cowboy_req.header/3 will be called with these args "user-agent", :req1, "null" and we define the return as: {:user_agent, :req2}.

Notice that again the req value is not important, but it's important to return a different req so you can track on which order it was executed as this is important to cowboy.

The second expectation is on JSEX.encode!/1 waiting for this argument [{"user-agent", :user_agent}] and return :json. The assertion will wait the tuple of 3 items:

  • Response body;
  • Request structure;
  • The state of the handler;

That's it! It's important to observe that this may not be enough if you have a really complicated handler. In this case you can write unit tests for each callback as I described and write integration tests without mocking anything and doing real HTTP requests.