Generating HTML
Prudence has excellent support for generating HTML by allowing you to embed programming language code in it, and allowing for superior performance and scalability via a sophisticated, fine-grained caching system.
Note that Prudence can also generate HTML at a "lower" level, via resources that return HTML representations.
While we're titling this section "Generating HTML," all of these methods can be used to generate any textual format, such as XML, JSON, etc.
Files
All files under /web/dynamic/ are assumed to be text files, and are mapped to URLs using the directory and file path, but without the filename extension. For example, the file "/web/dynamic/thread/list.html" would map to a URL like "http://mydomain.org/forum/thread/list/". You can also "capture" many URLs to be handled by a single file.
The filename's extension will be used to map the default MIME type for the file. For example, ".html" files will be "text/html", ".json" files will be "application/json", and ".xml" files will be "text/xml". You can easily change these default mapping via application configuration. Additionally, you can explicitly set the MIME type in a scriptlet via conversation.mediaTypeName.
The above is a very brief overview; see routing for an in-depth explanation of how files become URLs.
All text is implicitly assumed to be Unicode encoded as UTF-8. Use conversation.characterSet to explicitly translate it to other Unicode and non-Unicode character encodings.
Scriptlets
Programming language code can be embedded in your /web/dynamic/ files, making them all-powerful templates. Either "<% %>" or "<? ?>" delimiters can be used. Note, however, that you can only use one of either in the same file. The first type found will be the one used.
For historical reasons, these embedded bits of code are called "scriptlets." However, even though the word "script" tends to connote interpreted languages, scriptlets are usually compiled, not interpreted.
The standard output stream is the HTTP response. Thus, any "print"-like statements will be sent to the client.
In fact, behind the scenes, all non-scriptlet text is turned into code. It's a simple "print" of the non-scriptlet text into the output stream. Thus, you can freely mix scriptlet and non-scriptlet code:
<html>
<body>
<% for(var i = 0; i < 10; i++) { %>
This is line number <% print(i); %> out of 10.<br/>
<% } %>
</body>
</html>
The above translates into something like this:
print('<html>\n<body>\n');
for(var i = 0; i < 10; i++) {
print('This is line number ');
print(i);
print('out of 10.<br/>');
}
print('</body>\n</html>');
Newlines and Whitespace
It's important to remember that non-scriptlet text turns into "print" statements in your scriptlet language: these statements can have an effect on your code blocks, such as if/then blocks, loops, try/catch, closures, etc. Careless use of scriptlets can lead to syntax errors or, worse, to unexpected behavior.
For example, this JavaScript code will generate a syntax error:
<% var x = 0 %>You won't see this text!
The reason is that it produces the following mangled JavaScript code:
var x = 0 print('You won't see this text!')
Another example can show possibly unexpected results:
<% for(var i = 0; i < 10; i++) %>This line appears 10 times<br> And so does this line!<br>
If you expect the first line to appear 10 times and second line to appear once, you will be disappointed. The reason is that the newline is part of the "print", not part of your scriptlet code. It produces this JavaScript code:
for(var i = 0; i < 10; i++) print('This line appears 10 times<br>\nAnd so does this line!<br>')
Two ways to avoid this problem are:
-
Always put newlines in the beginning and end of your scriptlets, so that it's clear at a glance where statements begin and end:
<% var x = 0 %> You WILL see this text!
-
Alternatively, you can explicitly put end-of-statement marks, if your language supports them. For example:
<% var x = 0; %> You WILL see this text!
Python represents a special challenge in a scriptlet environment due to its reliance on significant whitespace for blocks. This means taking special care in mixing scriptlets with text. For example:
<% for x in range(0,10): %> This line appears 10 times<br> <% %> But this line appears once<br>
You'll notice that we put the end delimiter of the first scriptlet after an indentation. This tells Python that whatever follows (the "print" statement for the non-scriptlet text) will be in that for loop. We then used a scriptlet with only a newline to end the for loop.
A more complex example:
<%
for x in range(0,10):
for y in range(0,2):
%>
This line appears 20 times<br>
<%
%>
This appears every third line<br>
<%
%>
But this line appears once<br>
The trick here was using a scriptlet with an indentation to end the inner for loop, while keeping us in the outer for loop.
Tricky indeed, and definitely not easy to decipher at a glance. It may make more sense to use Python's "print" statement in such situations:
<%
for x in range(0,10):
for y in range(0,2):
print 'This line appears 20 times<br>'
print 'This appears every third line<br>'
%>
But this line appears once<br>
API
Beyond what's available to your programming language, scriptlets have access to Prudence's elaborate, powerful set of services via its API.
Code Libraries
Code files you put in your application's /libraries/ subdirectory can be accesses using your language's code inclusion mechanism. Here are examples in various languages. Let's assume that there is a file named "/libraries/util/data.*", which defines a function called getData():
-
Python:
from util.data import getData data = getData()
-
Ruby:
require 'util/data.rb' $data = get_data
-
Clojure:
(use 'util.data) (def data (get-data))
-
PHP:
require 'util/data.php'; $data = get_data();
-
JavaScript and Groovy do not have a built-in code inclusion mechanism, but they can use Prudence's document.execute API:
document.execute('/util/data/') data = getData()
Expression Scriptlets
A common idiom is to print out expressions interwoven with non-scriptlet text. The expression scriptlet, marked by an equal sign, can help you reduce clutter. It's equivalent to a simple "print" of the expression. For example, the above example could be shortened into:
<html>
<body>
<% for(var i = 0; i < 10; i++) { %>
This is line number <%= i %> out of 10.<br/>
<% } %>
</body>
</html>
Expression can be as elaborate as needed. For example, let's show line numbers above 5 in bold:
This is line number <%= i>5 ? '<b>' + i + '</b>' : i %> out of 10.<br/>
Comment Scriptlets
These scriptlets are ignored. Useful for adding comments, and for "commenting out" code temporarily during development:
<html> <body> <%# This whole scriptlet is ignored. var i = ugh... i don't know. will deal with it later. %> </body> </html>
Include Scriptlets and Fragments
You can include other files in-place using an include scriptlet, marked by an ampersand:
<html>
<body>
<% for(var i = 0; i < 10; i++) { %>
<%& '/lines/simple/' %>
<% } %>
</body>
</html>
The above example assumed a file named "/web/dynamic/lines/simple.html", with the following contents:
This is line number <%= i %> out of 10.<br/>
Just like an expression scriptlet is equivalent to a "print", an include scriptlet is equivalent to calling document.include. This, too, means that the include expression can be as elaborate as needed:
<%& '/lines/' + (i>5 ? 'emphasized/' : 'simple/') %>
Separating code into separate files that are then included is a very important technique for two reasons:
- It makes your application far more manageable by allowing you to reuse code blocks. A common use case is to have special "header.html" and "footer.html" files included by all pages in the application.
- In Prudence, each document has its own caching characteristics, allowing you to create elaborate caching schemes by splitting a file into smaller files. See caching and "Scaling Tips".
In the example above, our "/web/dynamic/lines/simple.html" file would be exposed via a URL. However, we do not intend this file to served in itself, only as a file to be included in other files.
To avoid external exposure, put files in /web/fragments/ instead of /web/dynamic/. Use is otherwise identical. For example, we'd move our file above to "/web/fragments/lines/simple.html". Files in /web/fragments/ are only available via include scriptlets.
An alternative to using /web/fragments/ is explicit URI hiding.
Custom Scriptlets
You can even create your own custom scriptlets using special handlers called scriptlet plugins, effectively extending Prudence's Scripturian-based templating engine.
Mixing Languages
By default, Prudence will assume your scriptlets to be in the source code of your Prudence flavor. For example, scriptlets in Prudence for JavaScript will be assumed to be JavaScript by default.
However, Prudence lets you use scriptlets of any language by adding the language name after the first scriptlet delimiter. For example, assuming we're using the JavaScript flavor:
<html>
<body>
JavaScript:<br/>
<%
// Here's some JavaScript code
for(var i = 0; i < 10; i++) {
print('Line ' + i + ' of 10<br/>')
}
%>
Python:<br/>
<%python
# Here's some Python code
for i in range(1, 10):
print 'Line %s of 10<br/>' % i
%>
</body>
</html>
Note that once you "switch" to another language, all subsequent scriptlets will be assumed to be of that language unless you explicitly set them to something else.
Expression and include scriptlets work the same way. For example:
<%&javascript 'lines/' + (i>5 ? 'emphasized/' : 'simple/') %>
Note that scriptlets in a specific language will only work if you have the the appropriate language JARs under your /libraries/ directory. The Prudence "Kitchen Sink" Edition comes with all languages supported by Prudence.
Templating Languages
You might think that mixing programming languages is a bad idea in general, and only (painfully) necessary for making use of legacy code. However, it can be a great idea if you consider that all Prudence flavors come with two high-performance templating languages, Velocity and Succinct.
By mixing languages, you can write all the more straightforward templating in a templating language, switching to full programming language scriptlets only when you need advanced features. For example, here we'll use JavaScript to set up our data, and then use Velocity to print it out:
<html>
<body>
<%
var country = myDatabase.fetchCurrentCountry()
conversation.locals.put('country', country)
%>
<h2>Famous Quotes:</h2>
<%velocity
<p>The rain in $conversation.locals.country stays mainly in the plain.</p>
## Here is a Velocity comment
%>
</body>
</html>
Note the use of conversation.locals above to pass data between scriptlets of various languages.
Not only is the Velocity code more readable and easier to manage, but it also performs better, is less prone to errors, and more secure. Prudence lets you use the right tool for the job.
Refer to the documentation of Velocity and Succinct to learn more about these templating languages.
In-Flow Scriptlets
One tiny hiccup to be aware of when mixing languages is that code blocks can not trivially span several scriptlets in multiple languages. For example, this won't immediately work:
<html>
<body>
<%javascript
for(var i = 0; i < 10; i++) {
conversation.locals.put('i', i)
%>
<%velocity
This is line $conversation.locals.i out of 10.<br/>
%>
<%javascript
}
%>
</body>
</html>
The reason is that, by necessity, Prudence runs scriptlets of different languages separately, in sequence. The language switch thus represents a syntactical boundary. The above would cause a syntax error, because we have a for-loop in the first scriptlet that isn't closed.
But, fear not, Prudence lets you solve this problem via the "in-flow" scriptlet, marked by a colon. Here's the exact same code as above, but with the Velocity scriptlet marked with an extra colon:
<html>
<body>
<%javascript
for(var i = 0; i < 10; i++) {
conversation.locals.put('i', i)
%>
<%:velocity
This is line $conversation.locals.i out of 10.<br/>
%>
<%
}
%>
</body>
</html>
This works! But how, you might wonder? Behind the scenes, the in-flow scriptlet is run from within the enclosing language. We thus never leave the enclosing language for the purposes of running through the file. Don't worry about performance here: in-flow scriptlets are compiled only once per file. Also, in-flow scriptlets are treated as regular scriptlets if there is no language switch, making them always safe to use if you're not sure.
One last note: in-flow scriptlets to not change the scriptlet language, so that we did not have to mark the closing bracked scriptlet as JavaScript.
Life
Files and all their scriptlets are only compiled once, when they are first requested by a client. The compiled code is kept in a cache, under your Prudence /cache/ directory. From then on, each request is handled by the compiled version, which is updated only if the original file changes. This cache is maintained even if you restart Prudence.
Unless a valid cache entry exists, the compiled code is executed anew for each request, essentially like a script. This is why global variables are not kept or shared between requests. Note that there's some minimal setup and cleanup involved in executing the code anew. It's very unlikely that this overhead would meaningfully affect performance, especially with caching in place, but you can avoid it if you use resources, which have a more static life cycle, instead of /web/dynamic/ pages.
Note that many threads might be running the same code concurrently: make sure you understand the concerns of concurrent programming, as detailed in "Managing State".
What if you edit the file? Prudence will automatically pick up your changes and recompile. This happens on the fly, while the application is running. Are you worried that this check would happen for every client request? You can easily control the minimum time interval in which Prudence assumes the file is unchanged, and will not check the file for validity.
An updated file would also cause cache entries produced by this file to be invalidated.
You can use document.addDependency to associate your own life to that of another file. If a dependency is updated, it will force your file to be reloaded.
Caching
You've chosen to generate HTML dynamically because you want requests to be able to have different results. However, sometimes you do not expect results, or parts of the results, to change very often. For example, a home web page might display the local temperature, but it would probably be good enough to update it every half hour, instead of per every request.
Depending on what your scriptlets are doing, dynamically generating a web page can be very costly, and could be a performance and scalability bottleneck for your application under heavy load. You don't want to waste resources to regenerate results that have not changed.
The solution is to cache results. Sometimes caching for even tiny time intervals can make a huge different in the ability of your application to handle load. For example, let's say that in order to fetch the current user count for our home page we need to query a database on the network. Without caching, every client request would result in a service query. Let's say our web page gets 100 hits per second. Caching our home page for a mere 5 seconds would throttle our database queries down to 1 every 5 seconds, vs. 500 every 5 seconds without caching. And users would get a current user count to 5 seconds old. It's a tolerable inaccuracy that generates enormous savings.
This was a trivial example. Truly scalable software requires smart caching everywhere, from the level of file and database access all the way to the generated results. For a full treatment of the topic, see "Scaling Tips".
Every /web/dynamic/ and /fragment/ document has its own caching interval, document.cacheDuration, which you can access and change in scriptlets. For example, here's the home page mentioned above:
<html> <% document.cacheDuration = 5 * 1000 // five seconds var userCount = myDatabase.getCurrentUsers().count() %> <body> <p>This is the home page!</p> <p><%= userCount %> users are currently logged in.</p> <p><%& '/temperature/' %></p> </body> </html>
Here is "/web/fragments/temperature.html":
<% document.cacheDuration = 30 * 60 * 1000 // half an hour var temperature = weatherServer.fetchCurrentTemperature() %> The current temperature is <%= temperature %> degrees.
Any file that includes the temperature fragment will use a cached result, up to half an hour old. The home page, however, is regenerated every 5 seconds, so that the user count is up to 5 seconds old. Prudence lets you finely control how caching works.
Note that you can change the cache duration dynamically. For example, you might want to increase it if you see that your machine's CPU load is too high, or you might even want to decrease it during "rush hours" where users expect to see especially up-to-date results.
The default cache key used by Prudence is sensible enough that it would not allow for cache conflicts while producing good results. However, Prudence provides many sophisticated tools for customizing cache keys for maximum efficiency. Start with the document.cacheKeyPattern API, and then take a look at cache key pattern handlers.
For further bandwidth and performance improvements with most clients, Prudence compresses your output by default, by setting conversation.encoding to the client's preferred encoding. Modern client processors, even on mobile devices, decompress fast enough that the network ends up being the performance bottleneck. The smaller the response, then, the better.
Because the cache key pattern automatically gets the encoding added to it, you will always be using the correctly encoded cache entry.
Compression does require some additional computation overhead. Note that if you are caching the output, it will be stored in the cache in its compressed state, such that subsequent cache retrievals would not involve any additional overhead. Additionally, if an uncompressed version already exists in the cache, it will be compressed, cached and used, for optimal efficiency. Thus, the total overhead would likely be insignificant for cached documents, while the bandwidth savings and performance benefits could be considerable. Caching and compression are good friends!
If there are no gains from compression in your application, you can disable Prudence's automatic use of the client's preferred encoding via the "com.threecrickets.prudence.GeneratedTextResource.negotiateEncoding" application global.
If you're running Prudence behind a proxy that already does compression, and are worried that you're compressing twice, then stop worrying: any decent proxy will know not to compress content that's already compressed.
Conditional Requests
HTTP clients, such as web browsers, store downloaded content in their cache, marking it with a modification date and tag according to HTTP headers in the response. Subsequent requests to the same URL will be "conditional," meaning that the client will tell the server which version it has. If the server still has the same version, then it returns an empty response with the 304 "not modified" HTTP status, letting the client know that it is safe to use its cached version. This saves both bandwidth and processing resources on the server, because it can avoid generating the representation.
If you're using document.cacheDuration, then conditional requests are already in effect for you by default. Prudence automatically sets the modification dates accordingly. You can disable this feature via the dynamicWebClientCachingMode setting.
Though clients rely on a local cache for conditional requests, you can provide them with additional caching directives. You can enable this via the the dynamicWebClientCachingMode setting.
Explicitly setting cache directives has an important side effect: most clients will not send conditional HTTP requests for the cached data until the cache entry expires. This allows you to save bandwidth and improves client performance, but at the expense of not being able to update the client upon changes. Use with care.
For even more control of these headers, consider using a resource instead.
Handling HTML Forms
Use of HTML's <form> tag is going out of fashion, as more powerful JavaScript-based approaches, such as "AJAX," are used instead to send data from the client to the server and to receive a response. Prudence has excellent support for "AJAX" via resources. Nevertheless, Prudence also fully supports HTML forms, via the conversation.form API.
Let's start with a simple form:
<html> <body> <form name="contact" method="POST"> <p>Your message to us:</p> <p><textarea name="content" cols="50" rows="15"></textarea></p> <p>An email address for us to send you a reply:</p> <p><input name="email" type="text"></input></p> <p><button type="submit">Send</button></p> </form> </body> </html>
As it stands, the above form will simply render again when the client clicks "send." Let's add a scriptlet to support the "POST" case:
<html>
<body>
<% if(conversation.request.method == 'POST') {
// TODO: write code that actually sends the message. :)
%>
Thank you for your message! You will be getting an answer to <%= conversation.form.get('email') %> soon.
<% } else { %>
<form name="contact" method="POST">
<p>Your message to us:</p>
<p><textarea name="content" cols="50" rows="15"></textarea></p>
<p>An email address for us to send you a reply:</p>
<p><input name="email" type="text"></input></p>
<p><button type="submit">Send</button></p>
</form>
<% } %>
</body>
</html>
Pretty simple!
And now, for handling uploaded files:
<html>
<body>
<% if(conversation.request.method == 'POST') { %>
Thank you for uploading your user avatar! It is temporarily stored at:
<%
var attachment = conversation.form.get('attachment')
print(attachment.file)
%>
<% } else { %>
<form enctype="multipart/form-data" name="upload" method="POST">
<p>Upload your user avatar:</p>
<p><input name="attachment" type="file"></input></p>
<p><button type="submit">Send</button></p>
</form>
<% } %>
</body>
</html>
Note that you will need to create an "/uploads/" subdirectory under your application's directory for uploads to work. See fileUploadBasePath to change this.
You might want to separate the form handler above into two fragments, one for displaying the form and the other for processing it. The main form could then include either fragment according to the value of conversation.request.method.
However, you can also use Prudence's capturing of URI patterns to automatically use either file without a main file to switch between the two. Here's an example application "routing.js":
document.execute('/defaults/application/routing/')
router.capture('/contact/', '/contact/{m}/')
We can then have two files, "contact/GET.html" and "contact/PUT.html" to handle each method for the same "contact/" URL.
A hybrid solution is to direct the form to a different URL, where you use a resource instead of a scriptlet to handle POST. Our example form tag would thus be:
<form name="contact" method="POST" action="/send/">
And then, under "/resources/send.js":
function handleInit(conversation) {
conversation.addMediaTypeByName('text/html')
}
function handlePost(conversation) {
var email = conversation.form.get('email')
return '<html><body>' +
'Thank you for your message! You will be getting an answer to ' + email + ' soon.' +
'</body></html>'
}
