The Case for REST
There's a lot of buzz about REST, but also a lot confusion about what it is and what it's good for. This essay attempts to convey REST's simple essence.
Let's start, then, not at REST, but at an attempt to create a new architecture for building scalable applications. Our goals are for it to be minimal, straightforward, and still have enough features to be productive. We want to learn some lessons from the failures of other, more elaborate and complicated architectures.
Let's call ours a "resource-oriented architecture."
Resources
Our base unit is a "resource," which, like an object in object-oriented architectures, encapsulates data with some functionality. However, we've learned from object-orientation that implementing arbitrary interfaces is a recipe for complexity: proxy generation, support for arbitrary types, marshaling, etc. And then we would need middleware to do all that heavy lifting for us. So, let's instead keep it simple and define a limited, unified interface that would be just useful enough.
From our experience with relational databases, we've learned that tremendous power can be found in "CRUD": Create, Read, Update and Delete. If we support just these operations, our resources will already be very powerful, enjoying the accumulated wisdom and design patterns from the database world.
Identifiers
Let's start with a way of uniquely identifying our resources. We'll define a name-based address space where our resources live. Each resource is "attached" to one or more addresses. We'll allow for "/" as a customary separator to allow for hierarchical addressing schemes. For example:
/animal/dog/3/ /animal/cat/12/image/ /animal/cat/12/image/large/ /animal/cat/12/specs/
In the above, we've allowed for different kinds of animals, a way of referencing individual animals, and a way of referencing specific aspects of these animals.
Let's now go over CRUD operations in increasing order of complexity.
Delete
"Delete" is the most trivial operation. After sending "delete" to an identifier, we expect it to not exist anymore. Whether sub-resources in our hierarchy can exist or not, we'll leave up to individual implementations. For example, deleting "/animal/cat/12/image" may or may not delete "/animal/cat/12/image/large".
Note that we don't care about atomicity here, because we don't expect anything to happen after our "delete" operation. A million changes can happen to our cat before our command is processed, but they're all forgotten after "delete." (See "update," below, for a small caveat.)
Read
"Read" is a bit more complicated than "delete." Since our resource might be changed by other clients, too, we want to make sure that there's some kind of way to mark which version we are reading. This will allow us to avoid unnecessary reads if there hasn't been any change.
Thus, we'll need our resource-oriented architecture to support some kind of version tagging feature.
Update
The problem with "update" is that it always references a certain version that we have "read" before. In some cases, though not all, we need some way to make sure that the data we expect to be there hasn't changed since we've last "read" it. Let's call this a "conditional update." (In databases, this is called a "compare-and-set" atomic operation.)
Actually, we've oversimplified our earlier definition of "delete." In some cases, we'd want a "conditional delete" to depend on certain expectations about the data. We might not want the resource deleted in some cases.
We'll need our resource-oriented architecture to support a general "conditional" operation feature.
Create
This is our most complex operation. Our first problem is that our identifier might not exist yet, or might already be attached to a resource. One approach could be to try identifiers in sequence:
Create: /animal/cat/13/ -> Error, already exists Create: /animal/cat/14/ -> Error, already exists Create: /animal/cat/15/ -> Error, already exists ... Create: /animal/cat/302041/ -> Success!
Obviously, this is not a scalable solution. Another approach could be to have a helper resource which provides us with the necessary ID:
Read: /animal/cat/next/ -> 14 Create: /animal/cat/14/ -> Oops, someone else beat us to 14! Read: /animal/cat/next/ -> 15 Create: /animal/cat/15/ -> Success!
Of course, we can also have "/animal/cat/next/" return unique IDs (such as GUIDs) to avoid duplications. If we never create our cat, they will be wasted, though. The main problem with this approach is that it requires two calls per creation: a "read," and then a "create." We can handle this in one call by allowing for "partial" creation, a "create" linked with an intrinsic "read":
Create: /animal/cat/ -> We send the data for the cat without the ID, and get back the same cat with an ID
Other solutions exist, too. The point of this discussion is to show you that "create" is not trivial, but also that solutions to "create" already exist within the resource-oriented architecture we've defined. "Create," though programmatically complex, does not require any additional architectural features.
Aggregate Resources
At first glance, handling the problem of getting lots of resources at the same time, thus saving on the number of calls, can trivially be handled by the features we've listed so far. A common solution is to define a "plural" version of the "singular" resource:
/animal/cats/
A "read" would give us all cats. But what if there are ten million cats? We can support paging. Again, we have a solution within our current feature set, using identifiers for each subset of cats:
/animal/cats/100/200/
We can define the above to return no more than 100 cats: from the 100th, to the 200th. There's a slight problem in this solution: the burden is on whatever component in our system handles mapping identifiers to resources. This is not terrible, but if we want our system to be more generic, it could help if things like "100 to 200" could be handled by our resource more directly. For convenience, let's implement a simple parameter system for all commands:
Read(100, 200): /animal/cats/
In the above, our mapping component only needs to know about "/animal/cats". The dumber our mapping component is, the easier it is to implement.
Formats
The problem of supporting multiple formats seems similar, at first glance, to that of aggregate resources. Again, we could potentially solve it with command parameters:
Read(UTF-8, Russian): /animal/cat/13/
This would give us a Russian, Unicode UTF-8 encoded version of our cat. Looks good, except that there is a potential problem: the client might prefer certain formats, but actually be able to handle others. It's more a matter of preference than any precision. Of course, we can have another resource where all available formats are listed, but this would require an extra call, and also introduce the problem of atomicity—what if the cat changes between these calls? A better solution would be to have the client associate certain preferences per command, have our resource emit its capabilities, with the mapping component in between "negotiating" these two lists. This "negotiation" is a rather simple algorithm to choose the best mutually preferable format.
This would be a simple feature to add to our resource-oriented architecture, which could greatly help to decouple its support for multiple formats from its addressing scheme.
Shared State
Shared state between the client and server is very useful for managing sessions and implementing basic security. Of course, it's quite easy to abuse shared state, too, by treating it as a cache for data. We don't want to encourage that. Instead, we just want a very simple shared state system.
We'll allow for this by attaching small, named, shared state objects to every request and response to a command. Nothing fancy or elaborate. There is a potential security breach here, so we have to trust that all components along the way honor the relationship between client and server, and don't allow other servers access to our shared state.
Summary of Features
So, what do we need?
We need a way to map identifiers to resources. We need support for the four CRUD operations. We need support for "conditional" updates and deletes. We need all operations to support "parameters." We need "negotiation" of formats. And, we need a simple shared state attachment feature.
This list is very easy to implement. It requires very little computing power, and no support for generic, arbitrary additions.
Transactions… Not!
Before we go on, it's worth mentioning one important feature which we did not require: transactions. Transactions are optional, and sometimes core features in many databases and distributed object systems. They can be extremely powerful, as they allow atomicity across an arbitrary number of commands. They are also, however, heavy to implement, as they require considerable shared state between client and server. Powerful as they are, it is possible to live without them. For example, we can implement complex atomicity schemes ourselves within a single resource. This puts some burden on us, but it does remove the heavy burden of supporting arbitrary transactions from our architecture. With some small reluctance, then, we'll do without transactions.
Let's Do It!
OK, so now we know what we need, let's go ahead and implement the infrastructure of components to handle our requirements. All we need is stacks for all supported clients, backend stacks for all our potential server platforms, middleware components to handle all the identifier routing, content negotiation, caching of data…
…And thousands of man hours to develop, test, deploy, and integrate. Like any large-scale, enterprise architecture, even trivial requirements have to jump through the usual hoops set up by the sheer scale of the task. Behind every great architecture are the nuts and bolts of the infrastructure.
Wouldn't it be great if the infrastructure already existed?
The Punchline
Well, duh. All the requirements for our resource-oriented architecture are already supported by HTTP:
Our resource identifiers are URLs. The CRUD operations are in the four HTTP verbs: PUT, GET, POST and DELETE. "Conditional" and "negotiated" modes are handled by headers, as are "cookies" for shared state. Version stamps are e-tags and timestamps. Command parameters are query matrices appended to URLs. It's all there.
Most importantly, the infrastructure for HTTP is already fully deployed world-wide. TCP/IP stacks are part of practically every operating system; wiring, switching and routing are part and parcel; HTTP gateways, firewalls, load balancers, proxies, caches, filters, etc., are stable consumer components; certificate authorities, national laws, international agreements are already in place to support the complex inter-business interaction. Best of all, this available infrastructure is successfully maintained, with minimal down-time, by highly-skilled independent technicians, organizations and component vendors across the world.
It's important to note a dependency and possible limitation of HTTP: it is bound to TCP/IP. Indeed, all identifiers are URLs: Uniform Resource Locators. In URLs, the first segment is reserved for the domain, either an IP address or a domain name translatable to an IP address. Compare this with the more general URIs (Uniform Resource Identifiers), which do not have this requirement. Though we'll often be tied to HTTP in REST, you'll see the literature attempting, at least, to be more generic. There are definitely use cases for non-HTTP, and even non-TCP/IP addressing schemes. In Prudence, it's possible to address internal resources with URIs that are not URLs; see internal APIs.
It's All About Infrastructure
The most important lesson to take from this exercise is the importance of infrastructure, something easily forgotten when planning architecture in ideal, abstract terms. This is why, I believe, Roy Fielding named Chapter 5 of his 2000 dissertation "Representational State Transfer (REST)" rather than, say, "resource-oriented architecture," as we have here. Fielding, one of the authors of the HTTP protocol, was intimately familiar with its deployment challenges, and the name "REST" is intended to point out the key characteristic of its infrastructure: HTTP and similar protocols are designed for transferring lightly annotated data representations, nothing more. "Resources" are merely logical encapsulations of these representations, depending on a contract between client and server. The infrastructure does not, in itself, do anything in particular to maintain, say, a sensible hierarchy of addresses, abitrary atomicity of CRUD operations, etc. That's up to your implementation. But, representational state transfer—REST—is the mundane, underlying magic that makes it all possible.
To come back to where we started: a resource-oriented architecture requires a REST infrastructure. In practice, the two terms become interchangeable.
The principles of resource-orientation can and are applied in many systems. The word wide web, of course, with its ecology of web browsers, web servers, certificate authorities, etc., is the most obvious model. But other core Internet systems, such as email (SMTP, POP, IMAP), file transfer (FTP, WebDAV) also implement some subset of REST. Your application can do this, too, and enjoy the same potential for scalability as these global, open implementations.
Does REST Scale?
Part of the buzz about REST is that it's an inherently scalable architecture. This is true, but perhaps not in the way that you think.
Consider that there are two uses of the word "scalable":
First, it's the ability to respond to a growing number of user requests without degradation in response time, by "simply" adding hardware (horizontal scaling) or replacing it with more powerful hardware (vertical scaling). This is the aspect of scalability that engineers care about. The simple answer is that REST can help, but it doesn't stand out. SOAP, for example, can also do it pretty well. REST aficionados sometimes point out that REST is "stateless," or "session-less," both characteristics that would definitely help scale. But, this is misleading. Protocols might be stateless, but architectures built on top of them don't have to be. For example, we've specifically talked about sessions here, and many web frameworks manage sessions via cookies. On the other hand, you can easily make poorly scalable REST. The bottom line is that there's nothing in REST that guarantees scalability in this respect. Indeed, engineers coming to REST due to this false lure end up wondering what the big deal is. We wrote a whole article for Scaling Tips, which is indeed not specifically about REST.
The second use of "scalability" comes from the realm of enterprise and project management. It's the ability of your project to grow in complexity without degradation in your ability to manage it. And that's REST's beauty—you already have the infrastructure, which is the hardest thing to scale in a project. You don't need to deploy client stacks. You don't need to create and update proxy objects for five different programming languages used in your enterprise. You don't need to deploy incompatible middleware by three different vendors and spend weeks trying to force them to play well together. Why would engineers care about REST? Precisely because they don't have to: they can focus on application engineering, rather than get bogged down by infrastructure management.
That said, a "resource-oriented architecture" as we defined here is not a bad start for—engineering-wise—scalable systems. Keep your extras lightweight, minimize or eliminate shared state, and encapsulate your resources according to use cases, and you won't, at least immediately, create any obstacles to scaling.
Prudence
Convinced? The best way to understand REST is to experiment with it. You've come to the right place. Start with the tutorial, and feel free to skip around the documentation and try things out for yourself. We're sure that you'll find it easy, fun, and powerful enough for you to create large-scale applications that take full advantage of the inherently scalable infrastructure of REST.