RESTers Gonna HATEOAS (hay-tee-us)

MR FIELDING, I PRESUME

REST is software design on the scale of decades: every detail is intended to promote software longevity and independent evolution. Many of the constraints are directly opposed to short-term efficiency. Unfortunately, people are fairly good at short-term design, and usually awful at long-term design. Most don’t think they need to design past the current release. There are more than a few software methodologies that portray any long-term thinking as wrong-headed, ivory tower design (which it can be if it isn’t motivated by real requirements).
– Roy Fielding

WHY REST

REST, as specified by Mr. Fielding in his original dissertation and subsequently in his various writings and talks, is an architectural style for distributed applications modeled after the World Wide Web. It’s distinguished from other APIs that provide an RPC mechanism using the HTTP methods. REST is more than just an RPC mechanism. It includes how to expose concepts and manipulate them in a manner that will allow the same APIs to evolve over time, yet continue to allow interoperability between applications.
If there were two traits to describe REST, they would probably be:

HTTP – honor HTTP mechanisms such as URIs, HTTP Methods, HTTP Status Codes, etc.
Evolvable thru Discovery – provide ways that a client can find out from the server what (resources) are available.

The goal, is to provide an API that can last a lifetime and to do that, a REST architecture defines how a client should walk over a set of resources exposed by another application. Just like the web, this interaction is stateless. When a user interacts with a web site, s/he isn’t required to know how to construct individual URLs to specific pages to get around and neither should a client application. Client applications of RESTful services can use content negotiation in the same manner that a web browser can negotiate with a server to retrieve resources in a form that it can render for the client. Often a web server will support more than one media type for a given resource.

REST will make use of same kind of caching mechanisms as found in the WWW and with similar linkage mechanisms to allow a client to walk over the resources and discover what is available and what it can do. This last point is important for building an API that can last over time. Clients need only to know how to walk this API to discover new resources that can be manipulated.

While REST focuses on designing an API that can last, many systems are designed for the immediate needs. Common design approaches advocate not even attempting to spend time on longer term objectives. I call this approach the:

“Don’t worry if you paint yourself in a corner, you can paint over your foot prints as you walk out the door”

Some of the REST principles create a tension with some of the various agile lean design approaches. Just be forewarned, there are many variations of RESTful and/or HTTP-RPC APIs out there, maybe even more than there are programmers.

HATEOAS

What’s HATEOAS? Hypermedia As The Engine of Application State – pronounced hay-tee-us (or any other pronunciation you might like to use. Think of the Linux pronunciation wars only worse. And remember if you pronounce it the way above, it’s an opportunity to feel superior.). So what is that? Well basically it recreates the www but for distributed systems rather than for web browsing.

With that, let’s get into the actual details shall we?

RESOURCES

Nouns typically have more staying power over processes, but there’s no real requirement that the resources be nouns. While a process may evolve, the basic concepts stay relatively unchanged thus the emphasis on constructing the majority of your API using domain objects.

Like SNMP, RESTful APIs expose resources with simple operations (HTTP Methods) that can be performed against them. Resources are much like platonic ideals, dancing happily in the heads of happy architects and engineers, distinct from actual representations (Media Types) used to describe them in whole or partially.

There are two kinds of resources: collections and individual resources.

URIS: CANONICAL AND ALIASES

Resources should have a unique canonical URI to identify them and one that applies only to that resource and is never reused for a different resource. Note: they can have more than one URI, but it’s generally encourage that Resources have one and only one canonical URI. For the purpose of this discussion, I’ll refer to these non-canonical, alternate URIs as being aliases.
Canonical URIs should be able to be permLinks, that is the URLs should be stable over time, while aliases may be transient or not. There are several reasons to use aliases:

For convenience. It may be that URLs that include the title are provided as convenience or URLs that include containment hierarchy that is subject to change at one point or another.
For distributed applications. A client make look up a resource using its canonical URL only to be temporarily redirected to a different URL where a server is currently hosting the resource. This is done for better performance etc.

If aliases are redirected to canonical URIs, then permanent redirects should be used. If canonical URIs are redirected to aliases in order to satisfy some temporary condition, then obviously temporary redirects should be used.

Naming styles: Plural and Singular Resource names

There are two or three basic naming styles currently being used out in the wild (well not including completely ad-hoc RPC style naming that goes on)

Plural names for all

This seems to becoming the most predominate way of naming resources.

/widgets – this is used to refer to the collection of all widgets in the system. Often there is a mechanism to page thru all of the widgets. Further, this resource may be used to create a new widget instance e.g. by sending a post to /widgets/new or to /widgets .
/widgets/{id} – refers to a specific widget. NOTE: there can be confusion between /widgets/new or /widgets/xxx and a specific instance /widgets/{id}.
2.1.1.2 Plural for collections/Singular for instances
This naming scheme keeps the URLs for collections and for individual instances completely separate to avoid any confusion. It looks something like:
/widgets – refers to a collection of widgets. POSTs are sent to /widgets to create a new widget or a PUT is done to /widget/{new-id} to create one.
/widget/{id}

Containment

It’s a question of how much of your containment hierarchy should be included in the URL. You can do something like:

/employers/{employerid}/user/{userid}

While the above is fine for aliases, it has a few issues for canonical URIs. Consider that we have user A with a URL of /employers/14/user/2.
If the user moves from employer/14 to another to say employer/155, so that the user will have a new URL /employer/155/user/33.
If no permanent redirect can be done, the URI /employer/14/user/2 will become invalid even though the resource still exists. A horrible problem occurs if the user id under the original employer is reused for user B. In that case, we have an old URL that is no pointing to a completely different URL. A cached URL to /employer/14/user/2 suddenly is associated with user B rather than user A. A lesser problem occurs when a user belongs to 2 employers, then there is no longer a single canonical URL to the resource, rather there now are 2.

A particularly bad canonical URL would be

/…/user/{username}

Or some other mutable property.

Collections: Querying, Sorting, Filtering, Pagination

For resources that are collections, it is common to provide a mechanism to querying, filtering and pagination.

OData: There is a comprehensive specification, OData, together with an implementation, OData4j, that allows for very generic querying, filtering and pagination compatibilities. The OData API uses path segments and query string parameters starting with a ‘$’ such as ‘$top’, ‘$offset’ to send queries and filtering and control pagination.

Custom paging. Custom paging generally uses query string parameters, e.g. ‘offset’ and ‘limit’, to page through the items in a collection and a query string parameter, e.g. ‘q’, to pass a search filter.

Range Header for paging. Some RESTful sites use a modified HTTP Byte range header to control pagination but this isn’t really part of the spec.

Resource Naming Canonical URI Anti-patterns

This list applies only to canonical URIs, not to aliases, which are simply convenient temporary ways of getting to a resource.

Reusing a URI for a different resource over time. This can break applications that have “bookmarked” a canonical resource in the past only to have it point to a completely different one in the future.
Using query string parameters as part of the resource identifier. NOTE: using query string parameters for content negotiation, filtering, searching etc is fine.
Using underscores. Stick to camelCase or hypens.
Using mutable properties in URL
Requiring clients to make use of out of band info to construct URIs. That is, passing back ‘ids’ for domain objects and requiring clients to ‘construct’ a URL based on some agreed upon contract like: Just take the id and put ‘/customers/’ in front of it to get the customer URI.
Not redirecting old URIs to their new locations when they change.

HTTP METHODS

Resources are manipulated via HTTP methods. An HTTP request is sent to the server with the request specifying the HTTP method, the URL of the resource, various request headers and potentially additional data in the Request body. HTTP Responses are returned by the server as the result of the particular HTTP method that was invoked. At a minimum, the response should have a status code. It may also have response headers and a body. Below shows the common HTTP methods used, together with a few of the common response codes (the list is NOT complete).

HTTP Method	RO	IDEM-POTENT	Description	Response Code
GET	yes	yes	Retrieves a resource (actually it gets a specific representation of a resource based on some of the Accept headers, other request headers and perhaps the URI). IDEMPOTENT, READOLY	200 if ok. 403 not authorized 404 not found 412 Precondition Failed
PUT	NO	yes	Modifies a resource. NOTE: can include the ETag from the last GET. If done, then this allows for an optimistic locking approach. If the ETag has changed for a resource, the servce can return a 412 (Precondition Failed). IDEMPOTENT	200 OK. 403 not authorized 404 not found. 412 precondition failed
POST	NO	NO	Does some operation, generally create. This is not IDEMPOTENT.	201 Created (used when POST creates a new resource). Should also include a location link to the new resource and optionally a response body with the new resource values. 200 OK (used when POST is doing an action) 404 resource not found
DELETE	NO	NO	Deletes the resource	200 OK 403 not authorize 404 Not found
OPTIONS	yes	yes	No one uses this. Idea is for the server to tell client what’s available. Like optional features implemented by server, actions, supported media types, etc. Note well defined. IDEMPOTENT, READONLY	200 OK 405 Method not allowed
HEAD	yes	yes	Same as GET, but with no response body. IDEMPOTENT, READONLY	200 ok 403 not authorized 404 not found
PATCH			Dangerous. Can leave an object in an inconsistent state since it is used to modify only a portion of an object’s state. More for mobile or limited bandwidth cases. TRICKY to get right
TRACE	yes	yes	Not really used

Method Antipatterns

The HTTP methods do not behave as expected. GET and HEAD should be safe (read-only) and idempotent. PUT should be idempotent. DELETE should remove the resource or fail.
Using only GET and POST.

REPRESENTATIONS

In the past, RESTful APIs supported at least JSON and XML, though most clients prefer JSON over XML. Refer to content negotiation for how a particular representation is chosen.

ETags

ETags are used to facilitate caching of responses locally. If implemented, an Etag is provided with the representation that can be used to see when its state changes.

A GET response for given URI with the same etag must have exactly the same data. This allows a client to do a GET request and set the request header ‘If-None-Match:’ to be the value of the etag in the previous GET response. If the data has not changed, the server should send back a 304 Not Modified response code.

CONTENT NEGOTIATION

Content negotiation is the name used for how the client can request different media types (representations) from the server. The specified way is to use one of the accept request headers.

Accept: followed by a comma separated list of types. E.g. application/json etc.
Accept-Language: the language to use.
Accept-Charset: the character set
Accept-Encoding: compress, gzip.

ANTI-PATTERNS

Ignoring (Mime) Media Types

LINKING: PUTTING THE ‘H’ IN HATEOAS

Links are used to point to other resources and provide mechanisms for paging through collections. Regardless as to whether you use links embedded in the media type or link response headers, links need to provide the href and the relationship. Additional attributes like the ‘title’ can be supported. Relationships are a bit tricky. There are standard relationships defined by
http://www.iana.org/assignments/link-relations/link-relations.xhtml

Custom ones may be provided, but they should be done in a way that doesn’t clash with the blessed link relations. Further there is a mechanism for specifying a document that describes the custom link relation. Take a look at iana.org.

LINKS IN RESPONSE BODY

Include the following:

{
	“id”: 123,
	“name”: “buster”,
	“links”: [
    {
		“href”: “/users/14”
		“rel”: “author”,
		“title”: “Terrance McNally”
    },
    …
     ]
}

LINK HEADERS

Also known as Web Links, these are response headers that have the href, rel and optionally other fields in them.

Link headers are used for non-text based media-types like ‘images’. Representations that are JSON or XML may use either link headers or embedded links. E.g. how the prev and next links are sent back when GETting a collection can be done using either method.

ANTI-PATTERNS

Ignoring hypermedia. Per HATEOAS, the way to give longevity to an API is to provide links so that a client can follow those links from one resource to another.
Not providing 1 or at most a small number of highlevel resources that have links which can be followed to get to the rest of the resources.

API VERSIONING

When asked how to do versioning, Fielding has stated DON’T. By that, you don’t just stop doing something one way, roll out a new “version” of the client and server and then continue on. Instead its all about having mechanisms in place to evolve the system. In other words, there’s no such thing as a breaking version. Old clients need to be written so that they can naturally handle the new changes without being rewritten.

Including version information in the resource URI negatively impacts interoperability, but that doesn’t mean that there can’t be new resources added.

WHEN TO ‘’VERSION’’

API changes occur when either the URI changes or when the responses change. These changes can be traditionally categorized as breaking and non-breaking changes, but we need to do handle them in such a way as to NOT break the clients. Typically a new version is needed only when a breaking change occurs. Response changes are typically confined to changes in the representations of the resource, that is, confined to changes in the media type.

URI versioning

URI changes should only be the result of changes to the associated resource types such as splitting or combining, otherwise those changes can be considered non-breaking by merely adding permanent redirects to the new URLs. Breaking changes are changes to old URIs that can’t or won’t be permanently redirected to a new URL.

An example of a breaking change is:

Version 1 User has a single Address
Version 2 User has one or more links to Address Resources.

The above still does not require a breaking change to the URIs. It will if the server is not able to redirect the /olduser/{id} to a new URL that can serve up the old user representation with just 1 address. If that is too much work, the client may have to be changed as a result of the resource changes.

Media Type versioning

Media Types AKA Mime types are used to describe a representation. Versioning of media types should be used for breaking changes. Using them for non breaking changes can lead to unnecessary churn on the client.

Versioning using custom media types. To allow for versioning to be done with accept headers, custom media types are used with version information in the types like below:

application/vnd.{compay}.{mediatype}-v1+json – this representation a specific JSON schema.
application/vnd.{compay}.{mediatype}.versionlist-v1+json
Returned when an invalid version is sent. This describes the allowed versions.
application/vnd.{compay}.{mediatype}-v1+xml – Returns custom XML.
application/vnd.{compay}.{mediatype}.versionlist-v1+xml – returns the supported XML schemas and versions.

Non breaking changes

A non-breaking change is considered to be the addition of an attribute/property such that the property can be ignored by the client. A reasonable implementation on the server side is to provide default values for this property when an object is created, and to leave the property unmodified by the client when PUTs are performed which do not have the property specified.

Breaking changes

A breaking change is one in which the client will have to be aware of the contract change. Examples of breaking changes are new ENUM values, removing properties or adding new properties that can’t be ignored. Breaking changes should be given a new rev of the media type.

Example Using Common Version Number

The server should return a response header on all responses indicating the current API version. (The last time that a ‘’breaking change’’ occurred).
Clients that depend upon the structure of the response (most clients), should keep the response version number and send it on all server requests. The server is required to either honor the version or to fail with the 501 Not Implemented status code.

Example using Custom Media Types

If the old internet media type was
```
	application/vnd.chl.accountdata-v1+json 
```
the new version would be
```
	application/vnd.chl.accountdata-v2+json
```
Only clients that accept the new v2 media type will get that version, others will get the old version.

ACTIVITY ID

A request id can be passed from process to process, thread to thread to follow the start of a processing of a request to its completion. By passing the id from server to server, logs can be correlated.
Use request header ‘CLH-Activity-Id’ with a UUID generated by the initial server.

SECURITY

Many sites are using OAuth2 or OAuth for security.

There is some confusion between 401 vs 403, but generally they mean:
- - - 401 => unauthenticated
    - 403 => unauthorized. No rights for client to do that.
CIRCUMVENTING BROWSER LIMITATIONS

It may be hard to control the request headers or the HTTP Method or both for some clients due to the browser or to various intermediaries (e.g. proxies) between the browser and the RESTful service. To get around this, that information may be passed in either request headers or as query string parameters.

For systems where request headers are supported, but only the standard GET and POST http method can be supplied, then a means of overriding the method must be used. Here are the 2 options:
- - - Method Override: Use request header ‘X-HTTP-Method-Override’ to set HTTP Method
    - Method Override: Use query string parameter to set HTTP method. E.g POST /customers/1?_method=PUT .
In cases where request headers can’t be sent, then content negotiation will have to sent using query string parameters or by using file extensions to the URL. E.g.
```
/customers/1?format=application/json
/customers/1.json
```
Versioning, request ids and other information may also be set using one of the above. E.g.
```
/customers/1?format=application/json&version=23-12-2004 
/customers.ver-23122004.json
```
JERSEY

TO BE DONE
Define a canonical implementation with some standard filters to extract activity ids from incoming requests.
Define a client to automatically pass on activity ids when making a request call.

Spring REST

TO BE DONE

ODATA

OData was initially developed by Microsoft and subsequently released under OASIS. OData supplies a lot of additional type metadata providing rich query, filtering and sorting. OData4J supports both client and server side processing of OData requests and responses.

MR FIELDING, I PRESUME

RELATED DOCUMENTS

WHY REST

HATEOAS

RESOURCES

URIS: CANONICAL AND ALIASES

Naming styles: Plural and Singular Resource names

Plural names for all

Containment

Collections: Querying, Sorting, Filtering, Pagination

Resource Naming Canonical URI Anti-patterns

HTTP METHODS

Method Antipatterns

REPRESENTATIONS

ETags

CONTENT NEGOTIATION

ANTI-PATTERNS

LINKING: PUTTING THE ‘H’ IN HATEOAS

LINKS IN RESPONSE BODY

LINK HEADERS

ANTI-PATTERNS

API VERSIONING

WHEN TO ‘’VERSION’’

URI versioning

Media Type versioning

Non breaking changes

Breaking changes

Example Using Common Version Number

Example using Custom Media Types

ACTIVITY ID

SECURITY

CIRCUMVENTING BROWSER LIMITATIONS

JERSEY

Spring REST

ODATA