Hosted by Three Crickets

Prudence
Scalable REST Platform
For the JVM





Prudence logo: bullfinch in flight

Caching

The State of the Art

Doing caching right is far from trivial: it's much more than just storing data in a key-value store, which is what most web platforms offer you.
Prudence's caching mechanism features the following:
But what's really great about Prudence is how easy it is to use these features: in most cases caching is pretty much automatic. When you need to customize, the API is clear and easy to use.

Server-Side Caching

Five of the caching APIs are in the "caching" namespace, and one is in "application".
For template resources, you may call these APIs anywhere on the page, but for manual resources they should be called in handleInit.

caching.duration

Specifies the duration of cache entries in milliseconds. Set this to a greater-than-zero value to enable caching on the current resource. The default is zero, meaning that caching is disabled.
You can set this value to a either a number or a string. For example, "1.5m" is 90000 milliseconds. Note, though, they when you read the value, it will always be numeric (a long integer data type).
Once enabled, every incoming request will have a cache key generated for it based on the cache key template, plus compression information. Prudence will attempt to fetch the cache entry from the cache, and if it's still valid, will display it to the user (this is called a "cache hit"). If there is no cache entry, or it's invalid, Prudence will run the resource as usual (this is called a "cache miss"), and then store a new cache entry via the key.
Compression is handled specially: if the requested compressed cache entry does not exist, then Prudence will attempt to fetch the uncompressed cache entry. If that exists, Prudence would simply compress it and store the compressed version so that compression could be avoided in the future. In the debug headers, this would appear as a "hit;encode" event. Likewise, when storing a new compressed cache entry (during a "miss"), Prudence actually stores both the compressed version as well as the uncompressed version.
See the API documentation for more details.
A Little Bit of Caching Goes a Long Way
Remarkably, even a very small cache duration of just a second or two can be immensely beneficial. It will ensure that if you're bombarded with a sudden upsurge of user requests to the resource, your application won't collapse. The cost is often very much worth it: having the "freshness" of your data being delayed by just a few seconds is usually not a big deal.
It's Not Always Worth It
It's important to remember that caching is not always faster than fully generating the page. Caching backends are generally very fast, but they still introduce overhead. So, just like in any scenario, avoid premature optimization and benchmark your resources to be sure that caching would indeed improve your performance and scalability.
Caching Forever?
You might think that your invalidation scheme is so perfect that there's no reason to ever have your cache entries expire. Well, think again: without a clear expiration time, your cache would continue growing forever. Finite durations thus allow for a way for the cache to recycle.

caching.tags

Tags are simple strings that you can associate with a resource's cache entries, which can then be used to invalidate all entries belonging to a particular tag. You may add as many tags as you wish: 
caching.tags.add('blog')
caching.tags.add('news.' + newsDate)
Note that tags are associated with all cache entries based on the resource, whatever their final cache key.
See the API documentation for more details.

caching.keyTemplate

The cache key template is a string with variables delimited in curly brackets that is cast into a cache key per request. The variables are elaborated in the chapter on string interpolation, and are essentially the same as those used for URI templates. However, Prudence also lets you install plugins to support your own specialized template variables. You can debug the cache key template by using the caching debug headers.
See the API documentation for more details.
Prudence's default cache key template is sensible enough for most scenarios: "{ri}|{dn}|{nmt}|{nl}|{ne}". You can change it in your settings.js. Let's break it down:
The above is a good cache key template, but you may want to modify it. Here are two common reasons:
Per-User Caching
A common scenario is for a resource to be generated different accordingly to the logged-in user. You would thus want to include a user identifier in the cache key. To do this, you would likely need to write a plugin to interpolate that identifier. You cache key template could then look something like "{ri}|{uid}|{MT}|{L}", where "uid" is handled by your plugin.
Caching Fragments
You might be including the same fragment in many pages, but the fragment in fact will be mostly identical. In this case, you can optimize by using a shorter cache key, such that the fragment would be cached only once for all inclusions. You would thus not want to use "ri". A simple example would be "{dn}|{MT}|{L}". This can also be used in conjunction with per-user caching: for example, if you want to cache the same fragment per-user, it would be "{dn}|{uid}|{MT}|{L}". Note that fragments are never compressed, so you don't need "{ne}".
Generally, creating the best key template involves a delicate balance between on the one hand making sure that differing data is indeed cached separately, while on the other hand making sure that you're not needlessly caching the same data more than once.

caching.keyTemplatePlugins

This powerful feature allows you to interpolate your own custom values into cache keys. While this does mean that some code will be run for every request, even for cache hits, it gives you the opportunity to write efficient, fast code that is used only for handling caching.
A common scenario requiring a key template plugin is to interpolate a user ID. We'd install it like so: 
caching.keyTemplatePlugins.put('uid', '/plugins/session/')
The above means that existence of a "uid" variable in a key template would trigger the invocation of the "/plugins/session/" library to handle it.
Actually, Prudence also allows you to install key template plugins by configuring them in your application's settings.js. In that case, the plugin would be installed for all resources: 
app.settings = {
	...
	code: {
		cacheKeyTemplatePlugins: {
			uid: '/plugins/session/'
		}
	}
}
The implementation of the plugin, however, would be the same however we install it. Our plugin would be in "/libraries/plugins/session.js": 
document.require('/sincerity/objects/')
​
function handleInterpolation(conversation, variables) {
	for (var v in variables) {
		var variable = variables[v]
		if (variable == 'uid') {
			var sessionCookie = conversation.getCookie('session')
			if (Sincerity.Objects.exists(sessionCookie)) {
				var session = getSession(sessionCookie.value)
				if (Sincerity.Objects.exists(session)) {
					conversation.locals.put('uid', session.getUserId())
				}
			}
		}
	}
}
​
function getSession(sessionId) {
	...
	return session
}
Implementation notes:
See the API documentation for more details.
An Optimization
In the above example, we are retrieving the session in order to discover the user ID, an operation that could potentially be costly. Consider that if we have a cache miss, then the session might be retrieved again in the implementation of the resource.
It's easy to optimize for this situation by storing the session as a conversation.local, such that it would be available in the resource implementation. We'd modify our above plugin code like so: 
if (Sincerity.Objects.exists(session)) {
	conversation.locals.put('session', session)
	conversation.locals.put('uid', session.getUserId())
}
Then, in our resource implementation, would could check to see if this value is present: 
var session = conversation.locals.get('session')
if (!Sincerity.Objects.exists(session)) {
	var sessionCookie = conversation.cookies.get('session')
	if (Sincerity.Objects.exists(sessionCookie)) {
		session = getSession(sessionCookie.value)
	}
}

caching.key

This is a read-only value, meant purely for debugging purposes. By logging or otherwise displaying it, you can see the cache key that Prudence would use for the current resource. Would is the key qualifier here: of course, your code displaying the cache key won't actually be run in the case of a cache hit.
Another way to see the cache key is to enable the caching debug headers.
See the API documentation for more details.

application.cache

You'll most likely want to use this API to invalidate a cache tag: 
application.cache.invalidate('blog')
See the API documentation for more details.

Backends

See the configuration chapter for a full guide to configuring your tiered caching backends. Prudence comes with many powerful options.

Client-Side Caching

Many HTTP clients, an in particular web clients, can cache results locally. Actually, HTTP specifies two different caching modes:

Automatic Client-Side Caching

Here's the good news: if you're using server-side caching, then client-side caching in conditional mode is enabled for you automatically, by default, for the cached resources. Moreover, Prudence will compute the expiration times accordingly and specifically per request. For example, if you are caching a particular resource for 5 minutes, and a client tries to access that resource for the first time after 1 minute has passed since the cache entry was stored, then the client will be told to cache the resource for 4 minutes. After those 4 minutes have passed, the client won't need to do a conditional HTTP request: it knows that it would need new data.
Automatic client-side caching applies to both template and manual resources.
Changing the Mode
If you wish, you may change the default mode from conditional to offline in your routing.js:
app.routes = {
	...
	{
		type: 'templates',
		clientCachingMode: 'offline',
		maxClientCachingDuration: '30m'
	}
}
Note that "maxClientCachingDuration" only has an effect in offline mode: it provides a certain safety cap against too-long cache durations. The default is -1, which means this cap is disabled.
Just make sure you understand the implications of offline mode: you will not be able to push changes to the client for the cached duration. You can also turn off client-side caching by setting "clientCachingMode" to "disabled".
Static Resources
You can add automatic client-side caching to static resources, too.

Manual Client-Side Caching

If you're not using Prudence's automatic caching, you can still benefit from client-side caching by using the APIs.
For conditional mode, you have the option of using modification timestamps and/or tags:
For offline mode:
Manual Ain't Easy
Note that though the APIs are very simple, leveraging them is not trivial, and may require you to design your data structures and subsystems with significant thought towards client-side caching.
For example, storing a modification timestamp within a single database entry is simple enough, but what if your final data is actually the result of a complex query using from several entries? You could potentially use the latest modification date of all of them: but, as you can see, calculating it can quickly get complicated and inefficient. Sometimes it might make sense to actually "bubble" modification dates upwards to all affected database entries as soon as you modify them: it would make your save operations heavier, but it could very well be worth it for greater scalability and an improved user experience.
In some cases, calculating a tag might be less costly than keeping track of modification timestamps. For some data structures, tags may even be provided for you as a byproduct of how they work: key hashes, serial IDs, checksums, etc., are all great candidates for tags.
Optimizing the Server
Conditional mode can improve the client experience, but it can improve the server experience, too.
Prudence, using a great feature of the Restlet library, lets you create the conditional HTTP headers and return them to the client without generating the response. Thus, only if the conditional request continues would your response generation code be called. This feature is internally used by Prudence's automatic caching, but you can also use it yourself in manual resources, using the "handleGetInfo" entry point.
Here's an example—not the most efficient one, but it will demonstrate the flow: 
function handleGetInfo(conversation) {
	var id = conversation.locals.get('id')
	var data = fetchDataFromDatabase(id)
	conversation.locals.put('data', data)
	return data.getModificationTimestamp()
}
​
function handleGet(conversation) {
	var data = getData(conversation)
	conversation.modificationTimestamp = data.getModificationTimestamp()
	return Sincerity.JSON.to(data)
}
​
function getData(conversation) {
	var data = conversation.locals.get('data')
	if (!Sincerity.Objects.exists(data)) {
		var id = conversation.locals.get('id')
		data = fetchDataFromDatabase(id)
	}
	return data
}
​
function fetchDataFromDatabase(id) {
	...
}
As you can see, we're storing the fetched data in a conversation.local, so that if handleGet is called after handleGetInfo, we would not have to access the database twice.
What have we accomplished in this example? Not that much: all we've done is avoided JSON serialization for those conditional requests that stop at handleGetInfo. A worthwhile little optimization, to be sure, but not one with very dramatic effects. It might be more effective in cases in which we had other heavy processing in handleGet that could be avoided.
The handleGetInfo trick really shines when you have a shortcut to accessing the modification date or the tag. Consider as a common example the a way a filesystem works: you can fetch the file modification date with one system API call, without actually opening the file for reading its contents, which would of course be a much costlier operation. Using handleGetInfo with that API would be able to affect a crucial (even necessary!) optimization. Indeed, that's how static resources work internally.
But how would you implement this with a database server? Most database servers don't allow for such shortcuts: sure, you could only fetch the modification date column from a row for handleGetInfo, but it would be inefficient if soon after you would also need to fetch the rest of the columns. It would be more efficient to just fetch the entire row at once, and so you're back to our non-dramatic optimization from before.
What you could do, however, is cache only the modification dates separately in a specialized backend that is much lighter and faster than the database server, for example Hazelcast or memcached. Here's how it might look: 
function handleGetInfo(conversation) {
	var id = conversation.locals.get('id')
	var modificationTimestamp = fetchTimestampFromCache(id)
	return Sincerity.Objects.exists(modificationTimestamp) ? modificationTimestamp : null
}
​
function handleGet(conversation) {
	var data = fetchDataFromDatabase(conversation)
	conversation.modificationTimestamp = data.getModificationTimestamp()
	storeTimestampInCache(id, data.getModificationTimestamp())
	return Sincerity.JSON.to(data)
}
​
function fetchDataFromDatabase(id) {
	...
}
​
function fetchTimestampFromCache(id) {
	return conversation.distributedGlobals.get('timestamp:' + id)
}
​
function storeTimestampInCache(id, timestamp) {
	conversation.distributedGlobals.put('timestamp:' + id, timestamp)
}
As you can see, the optimization won't be effective unless the cache is warm. Thus, to make it truly effective, you would need a special subsystem to warm up the cache in the background…
Welcome to the world of high-volume web! The solutions for massive scalability are rarely trivial. While Prudence can't provide you with automation for every scenario, at least it provides you with the tools on which you can build comprehensive solutions. With careful planning, you can go very far indeed.
In summary, before you go ahead and provide a handleGetInfo entry point for every resource you create, consider:
  1. It could be that you don't need this optimization. Make sure, first, that you've actually identified a problem with performance or scalability, and that you've traced it to handleGet on this resource.
  2. It could be that you won't gain anything from this optimization. Caches and other optimizations along the route between your data and your client might already be doing a great job at keeping handleGet as efficient as it could be. If not, improving them could offer far greater benefits overall than a complex handleGetInfo mechanism.
  3. It could be that you'll even hurt your scalability! The reason is that an efficient handleGetInfo implementation would need some mechanism in place to track of data modification, and this mechanism can introduce overhead into your system that causes it to scale worse than without your handleGetInfo.
See "Scaling Tips" for a thorough discussion of the problem of scalability.

Bypassing the Client-Side Cache

A devilishly useful aspect of client-side caching is that the cache key is the entire URL, including the query. This means that by simply adding a query parameter (which you otherwise ignore in your server-side handling), you can force the client to fetch new data, even when using offline mode caching for that resource.
Of course, you can't use this trick unless you can control the URLs which the client uses. Luckily, this is exactly what you can do in HTTP: see the static resources guide for a comprehensive discussion.

Two Client-Side Caching Strategies

The default Prudence application template is configured with minimal client-side caching, which is suitable for development deployments. However, once you are ready to move your application to production or staging, you will likely want a more robust caching strategy.
We will here present two common strategies, and discuss the pros and cons of each. They are intended as polar opposites, though you may very well choose a strategy somewhere in between.
Paranoid: Short-Term Caching
This is a great strategy if you're not feeling very confident about managing caching in your application logic. Perhaps you have too many different kinds of pages requiring different caching strategies. Perhaps you can't maintain the strict discipline required for more aggressive caching, due to a quickly changing application structure ("agile"?) or third-party constraints.
If you're in that boat, short-term caching is recommended over no caching at all, because it would still offer better performance and scalability. Because caching is short-term, any mistakes you make won't last for very long, and can quickly be fixed.
How short a term depends on two factors: 1) usage patterns for your web site, and 2) the content update frequency. For example, if a user tends to spend about an hour browsing your site, then a one-hour caching duration makes sense: the client would only have a slightly slower page load at the beginning of the visit.
Our strategy would then be to:
Here's an example routing.js: 
app.routes = {
	'/*': [
		...
		'manual', // clientCachingMode: conditional
		'templates', // clientCachingMode: conditional
		{
			type: 'cacheControl',
			mediaTypes: {
				'image/*': '1h',
				'text/css': '1h',
				'application/x-javascript': '1h'
			},
			next: {
				type: 'less',
				next: 'static'
			}
		}
		...
Confident: Long-Term Caching
For long-term caching to work, you must have good systems in place for bypassing the cache when necessary:
Our strategy would then be to:
Here's an example routing.js: 
app.routes = {
	'/*': [
		...
		'manual', // clientCachingMode: conditional
		{
			type: 'templates',
			clientCachingMode: 'offline'
		},
		...
		{
			type: 'cacheControl',
			mediaTypes: {
				'image/*': 'farFuture',
				'text/css': 'farFuture',
				'application/x-javascript': 'farFuture'
			},
			next: {
				type: 'less',
				next: 'static'
			}
		},
		...

The Prudence Manual is provided for you under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The complete manual is available for download as a PDF.

Download manual as PDF Creative Commons License