Programming
Prudence, and the underlying Sincerity bootstrapping tool, allow you to use your choice among several different programming languages on top of the JVM, and moreover they provide you with context-specific, well-documented APIs per relevant execution environment.
Your current programming skills should transfer readily enough, however some principles might be new to you:
- The main execution environments you will be working with in Prudence are inherently multi-threaded, meaning that you must have some basic knowledge of concurrent programming. The API does its best to help you, but it's still crucial that you understand it.
- If you've never used the programming language on the JVM before, you'll might find some pitfalls in terms of interaction with the Prudence and JVM APIs, and these vary per programming language engine. Usually, this has to do with differences in types: strings, numbers, arrays/lists, etc. Additionally, some JVM implementations differ in small but important ways from the reference implementations. Usually, things will work as expected: if not, you should refer to the documentation for the specific language engine.
Scripturian is the magical library that enables much of Prudence and Sincerity: it abstracts away the specifics of the programming language you're using, thus allowing you to transparently switch between programming languages and engines. It compiles your changes on the fly, caches the bytecode, parses templates into code, provides access to the environment's APIs, and does it all while optimizing for high-performance concurrency.
You don't have to understand how Scripturian works to use Prudence, however it's useful to remember that usually Prudence is ignorant as to what programming language you're using: that's all handles by Scripturian.
APIs
Prudence provides you with an especially rich set of APIs. They come in three categories:
- The Prudence Core APIs are multilingual: they are implemented via standard JVM classes that can be called from all supported programming languages: JavaScript, Python, Ruby, PHP, Lua, Groovy and Clojure. Indeed, the entire JVM standard APIs can be access in this manner, in addition to any JVM library installed in your container (under "/libraries/jars/").
- Prudence JavaScript APIs: these JavaScript wrappers over the Prudence Core APIs make using them a bit easier with JavaScript. Future versions of Prudence may provide similar friendly wrappers for the other supported languages (please contribute!). Until then, non-JavaScript programmers shouldn't be too jealous: there's nothing that these wrappers can do that you can't do with the core APIs, and it's easy to look into the source code in case you want to duplicate the same functionality in your language of choice.
- Sincerity JavaScript APIs: most of the supported programming languages have rich standard libraries as well as an ecology of external libraries. JavaScript, however, stands out for having a very meager standard library. To fill in this gap, Sincerity comes with a useful set of JavaScript libraries. Some of these are written pure JavaScript, offering new and useful functionality, while others provide JavaScript-friendly wrappers over standard JVM libraries.
For the sake of coherence and convenience all these APIs are documented together online, with direct links to the source code. The entire documentation uses the JavaScript calling convention, even for the multlingual APIs, however it should be trivial to use the calling convention of your language of choice.
You may be further interested in looking up Prudence's low-level API, which is also fully documented online. Also, sometimes the best documentation is the source code itself.
The development team spends a lot of time meticulously documenting the APIs. Please send us a bug report if you find a mistake, or think that the documentation can be improved!
Prudence Core APIs
These core APIs can be used by any supported programming language. See calling conventions for instructions on how to use these APIs from your language of choice.
The APIs consist of four namespaces (objects) that are defined as global variables:
- application: general application services, such as global state, logging and background tasks
- conversation: services related to the current request-and-response; available for manual and template resources, and also for filters
-
document: actually incorporates two kinds of services:
- access to other documents, such as executing or requesting them
- low-level access to the current document
- executable: rarely used, low-level access to Scripturian
JavaScript Libraries
The APIs are only available for JavaScript.
To use them, you must "document.require" them according to their URI, and then via their namespaces. For example:
document.require( '/sincerity/json/', '/prudence/tasks/) println(Sincerity.JSON.to({hello: 'world'}))
You can find their source code in the "/libraries/scripturian/" directory of your container.
These libraries are intended to fill in for the lack of a rich standard library for the JavaScript language. They are general-purpose and not specific to Prudence. If you've installed Prudence using Sincerity, then you will find them in your Sincerity installation rather than in your Prudence container (under the "/libraries/scripturian/" directory).
- /sincerity/calendar/: useful helpers for JavaScript Date objects
- /sincerity/classes/: a rich and powerful object-oriented programming (OOP) facility, supporting inheritance, private data and methods, and automatic constructors
- /sincerity/cryptography/: JavaScript-friendly wrappers over the JVM's strong cryptographic libraries, allowing for digests/hashing, encryption/decryption, and cryptographically-strong random number generation
- /sincerity/files/: high-performance reading/writing of files, as well as filesystem operations
- /sincerity/iterators/: standardized access to streaming data structures, with plenty of utility classes for stream manipulation
- /sincerity/json/: high-performance, extensible JSON/JavaScript conversion
- /sincerity/jvm/: conversions between JavaScript and JVM data types
- /sincerity/localization/: unit conversions and human-readable formatting of dates and times
- /sincerity/lucene/: JavaScript-friendly wrappers over the Lucene search engine
- /sincerity/mail/: send emails using JavaScript-friendly wrappers over JavaMail with support for message templates
- /sincerity/objects/: many important enhancements to and utilities for working with standard JavaScript objects, such as dicts, arrays and strings
- /sincerity/platform/: utilities specific to the underlying JavaScript engine (Nashorn or Rhino)
- /sincerity/templates/: straightforward and powerful string interpolation
- /sincerity/xml/: parsing and generation of XML
These are JavaScript-friendly wrappers over the Prudence Core APIs, as well as other special uses for JavaScript.
- /prudence/logging/: adds automatic support for string interpolation, exception logging, conditional logging, and other convenient utilities; see logging for more details
- /prudence/resources/: a rich set of utilities for generated and parsing web data
- /prudence/tasks/: JavaScript-friendly spawning and scheduling of background tasks
We're listing these libraries separately, because they are specifically meant to be used for application configuration.
- /sincerity/annotations/: allows you to easily gather specially marked annotations in text files, which can include your source code and templates; specially designed to integrate well with scriptlet comments
- /sincerity/container/: access to the Sincerity container into which Prudence has been installed
- /prudence/lazy/: see lazy initialization for more details
- /prudence/setup/: handles the parsing of routing.js and settings.js
Calling Conventions by Language
conversation.redirectSeeOther('http://newsite.org/') caching.onlyGet = true application.globals.put('name', 'example') // Nashorn: application.globals['name'] = 'example'
If you're using JVM 8, with Nashorn as your JavaScript engine, then you can treat JVM maps as dictionaries: "application.globals['myapp.data.name']". However, for JVM 7 and Rhino you must use the get- and put- notation: "application.globals.get('myapp.data.name')". In order to support both engines properly, we recommend using the more cumbersome format used by Rhino.
conversation.redirectSeeOther('http://newsite.org/') caching.onlyGet = True // Jepp: caching.setOnlyGet(True) application.globals['name'] = 'example' // Jepp: application.getGlobals().put('name', 'example')
If you're using the Jepp engine, rather than the default Jython engine, you will need to use get- and set- notation to access attributes. For example, use "application.getArguments()" to access "application.arguments" in Jepp.
$conversation.redirect_see_other 'http://newsite.org/' $caching.only_get = true $application.globals['name'] = 'example'
Prudence's Ruby engine, JRuby, conveniently lets you use the Ruby naming style for API calls. For example, you can use "$application.get_global" instead of "$application.getGlobal".
$conversation->redirectSeeOther('http://newsite.org/'); $cahing->onlyGet = TRUE; $application->globals['name'] = 'example';
conversation:redirectSeeOther('http://newsite.org/') caching:setOnlyGet(true) application:getGlobals():put('name', 'example')
You will need to use the get- and set- notation to access attributes. For example, you must use "conversation:getEntity()" to access "conversation.entity".
conversation.redirectSeeOther('http://newsite.org/') caching.onlyGet = true application.globals['name'] = 'example'
(.. conversation redirectSeeOther "http://newsite.org/") (.setOnlyGet caching true) (.. application getGlobals (put "name" "example"))
You will need to use get- and set- notation to access attributes. For example, use "(.getArguments application)" to access "application.arguments". You can also use Clojure's bean form, for example "(bean application)", to create a read-only representation of Prudence's API services.
Entry Points
Prudence sometimes treats your code as its API: for this, it requires you to implement specific "entry points" in your code. What these are, exactly, differs per programming language, as detailed below.
Uses global functions, with the camel-case naming convention:
function handleGet(conversation) { return 'Hello, world!' }
Uses global functions, with the lowercase-with-underscores naming convention:
def handle_get(conversation): return 'Hello, world!'
Uses global methods, with the lowercase-with-underscores naming convention:
def handle_get conversation return 'Hello, world!' end
Uses global functions, with the lowercase-with-underscores naming convention:
function handle_get($conversation) { return 'Hello, world!'; }
Uses global functions, with the lowercase-with-underscores naming convention:
function handle_get (conversation) return 'Hello, world!' end
Uses closures tied to global variables (there are no global methods in Groovy), with the camel-case naming convention:
handleGet = { conversation -> return 'Hello, world!' }
Uses functions in the current namespace, with the lowercase-with-dashes naming convention:
(defn handle-get [conversation] "Hello, world!")
State and Scope
Prudence is designed to allow for high concurrency and scalability, while at the same time shielding you from the gorier details. However, it's critical that you understand how to access and manage state and scope with Prudence.
application.globals
The application.globals and application.getGlobal APIs are essential for sharing state across the application: all application code can access it using the same API, whether it's a manual resource, a template resource, a filter or a background task. Because this means that application.globals may be accessed simultaneous by multiple threads, it's important that you understand how to use them concurrently.
Another important use for application.globals is configuration: you can specify configuration settings as app.globals in your settings.js, and then easily access them as application.globals one the application is running.
Be very careful with global variables in Prudence: their scope is more limited and more temporary than you might think.
Their behavior is actually a very different depending on the execution environment of your code:
- Manual resources and filters: Your global values will persist as long as the code is not recompiled. Anything that will trigger recompilation of your code will also involve recreating a new global context for it, discarding all previously used globals. In between these recompilations, your globals will persist, but it's important to remember that they will be shared between all threads, and as thus should be accessed with concurrency techniques.
- Template resources: These are made of programs executed from beginning to end, and as such every execution uses its own global context. This means that globals are really "local" to the request, like conversation.locals. But it also means that you don't have to worry about concurrency when using them.
A good rule of thumb is to assume that global variables never persist beyond a request, and to use application.globals whenever persistence is required.
Note for Clojure: Clojure doesn't have true global variables: instead all Vars are bound to a particular namespace. On the other hand, all namespaces are global to JVM: there are no thread-limited scopes. In order to allow for threads to have separate global scopes, Scripturian creates on-the-fly namespaces when necessary (this is very lightweight), each with a unique name. Thus, Vars in Clojure end up behaving exactly the same as those in other programming languages.
application.sharedGlobals
The application.sharedGlobals and application.getSharedGlobal APIs have the same scope as application.globals, except they are shared between all running applications.
Generally, it's not such a good idea to create interdependencies between applications, however there are cases where it is useful:
- If all running applications are connecting to the same database (or another external resource), it can be a good idea to have them share the same connection pool. This allows you to centrally manage the connections on the JVM.
- You can use shared globals to pass messages between running applications: the shared value can be a LinkedBlockingQueue or other thread-safe message-passing mechanism.
- Shared globals can be used for system-wide configuration settings. A good way to set shared globals is via a custom service.
If you find yourself using application.sharedGlobals a lot, ask yourself if your code would be better off encapsulated as a single application: remember that Prudence has powerful URI routing, support for virtual hosting, etc., letting you easily have one application work in several sites simultaneously. In short, there might be a better architecture for what you're trying to do.
Note for Clojure: Actually, Clojure namespaces are identical in scope to application.sharedGlobals (all Prudence applications running in the same JVM share the same Clojure namespaces), so you might prefer to use them because they are more idiomatic. Still, application.sharedGlobals can be useful if you want to share globals with other programming languages.
application.distributedGlobals and application.distributedSharedGlobals
The application.distributedGlobals, application.getDistributedGlobal, application.distributedSharedGlobals and application.getDistributedSharedGlobal APIs work similarly to application.globals and application.sharedGlobals, except that they are shared between all members of a cluster.
See the clusters chapter for more information.
executable.globals
The executable.globals and executable.getGlobal APIs have a similar scope to application.sharedGlobals, except that they can be accessed by any code running on the JVM using Scripturian's GlobalScope API. They're useful if you need to share state with non-Prudence code.
conversation.locals
The conversation.locals are not "local" in the same way that code scope locals are. The term "local" here should be read as "local to the conversation." They are indeed "global" in the sense that they can be accessed by any function in your code, but are "local" in the sense that they persist only for the duration of the user request. (Compare with thread locals, which are also "local" in a specific sense.)
Their primary use is for sharing state among code along the route: for example, both your resource code and filters will have access to the same conversation.locals per request. It's this essential feature that leads them to being used in many ways throughout Prudence:
- URI template variables are captured into conversation.locals.
- They can be interpolated into target URIs
- They are used for inversion of control
- They are used to inject custom values into cache keys
- Template blocks are captured into conversation.locals
Sometimes, your language's global variables function identically to conversation.locals. Consider this template resource:
<html><body> <% var name = 'Rambo'; %> <%& '/hello/ %> </body></html>
Our "/libraries/includes/hello.t.html" can look like this:
<p>Hello, <%= name %>!</p>
As you can see, the global variable "name" is indeed shared between these two fragments, and there's no advantage to using conversation.locals instead.
But you might have to use conversation.locals if:
- …you need to share state with a filter
- …you are mixing scriptlets written in different languages
Synchronization
Sometimes efficient techniques for handling concurrency, detailed below, can't be implemented, and the only solution is to ensure single-thread access via synchronization.
Synchronization is very bad for scalability: it works against many of the advantages of using multiple threads, processes and nodes. However, in some situations it may be preferable to the alternatives. Use it responsibly. Especially, you want to avoid using it in request threads: it makes a bit more sense for background task threads.
Prudence makes synchronization easy via the application.getLock, application.getSharedLock and application.getDistributedSharedLock family of APIs. Here's an example of how to use them:
var lock = application.getLock('services.remote') lock.lock() try { doSomethingAtomicallyWithRemoteService() } finally { lock.unlock() }
Note our use of try/finally: it's in order to guarantee that the lock is released, even if an exception is thrown from our code. Unreleased locks will cause your threads to hang.
Also note that locks will stay in memory, unless they are distributed. They take very little memory, but if for some reason you are generating many locks over time, you may want to cull them via the application.locks or application.sharedLocks APIs.
Concurrency
Though application.globals, application.sharedGlobals, application.distributedGlobals and executable.globals are all thread safe, it's important to understand how to use them properly.
Note for Clojure: Though Clojure goes a long way towards simplifying concurrent programming, it does not solve the problem of concurrent access to application.globals. You still need to read this section!
This code might seem OK to you, but it's subtly broken:
function getConnection() { var connectionPool = application.globals.get('database.pool') if (!Sincerity.Objects.exists(connectionPool)) { connectionPool = createPool() application.globals.put('database.pool', connectionPool) } return connectionPool.getConnection() }
The problem is that in the short interval between comparing the value in the "if" statement and setting the global value in the "then" statement, another thread may have already set the value. Thus, the "connectionPool" instance you are referring to in the current thread would be different from the "database.pool" application.global used by other threads. The valye is thus not truly shared! In some cases, this would only result in a few extra, unnecessary connection pools being created. But in some cases, when you rely on the uniqueness of the global, this can lead to subtle bugs that will appear only under high concurrency.
Newcomers to concurrent programming tend to think that these kinds of bugs are very rare: another thread would have to set the value exactly between our "if" and our "then." This is wrong: if your application has many concurrent users, and your machine has many CPU cores, it can actually happen quite frequently. And, even if rare, your application has a chance of breaking if just two users use it at the same time. This is not a problem you can gloss over, even for simple applications.
Here's a fixed, concurrently safe version of the above code:
function getConnection() { return application.getGlobal('database.pool', createPool()).getConnection() }
The application.getGlobal call is an atomic compare-and-set operation, which guarantees that the returned value is a unique instance.
There are two things worth noting:
- The fixed code is much shorter than the broken code! This is due to the application.getGlobal API, which internally actually does several operations for you.
- You can see that createPool is always called, even if the internal compare-and-set of application.getGlobal fails. This means that in some cases we will be creating a database connection pool and then immediately discarding it. This is a waste, but it's also a security issue: under very high concurrency, we might be creating a lot of these unnecessary connection pools, which could in fact lead to an overload on our database server. We thus might be vulnerable to denial of service (DoS) attacks.
We can significantly reduce the number of times createPool is called by checking to see if the application.global is already set:
function getConnection() { var connectionPool = application.globals.get('database.pool') if (!Sincerity.Objects.exists(connectionPool)) { connectionPool = application.getGlobal('database.pool', createPool()) } return connectionPool.getConnection() }
With this code, we might still have createPool called multiple times if the function is called concurrently while the application.global is still unset. The problem will disappear as soon as the application.global is set, but until then we still have a window of vulnerability.
The only way to guarantee that createPool is not called more than once is make the entire operation atomic by synchronizing it:
function getConnection() { var lock = application.getLock('database.pool') lock.lock() try { var connectionPool = application.globals.get('database.pool') if (!Sincerity.Objects.exists(connectionPool)) { connectionPool = application.getGlobal('database.pool', createPool()) } return connectionPool.getConnection() } finally { lock.unlock() } }
Not only is the code above complicated, but synchronization introduces performance penalties. It's definitely not a good idea to blindly apply this solution.
Concurrent programming is non-trivial and always involves weighing the pros and cons of various solutions for specific situations. So, hybrid approaches are sometimes the best: choose the right solution according to the context.
Finally, then, here's a version of the above code that will allow us to select if we want to use the lock or not:
function getConnection(safe) { if (safe) { var lock = application.getLock('database.pool') lock.lock() } try { var connectionPool = application.globals.get('database.pool') if (!Sincerity.Objects.exists(connectionPool)) { connectionPool = application.getGlobal('database.pool', createPool()) } return connectionPool.getConnection() } finally { if (safe) { lock.unlock() } } }
Execution Environments
This section serves as a summary for advanced programmers who are curious about the differences between the many Scripturian-based execution environments available in Prudence.
Programs vs. Entry Points
These two types of execution environment are very different in terms of programming, scopes and threads.
Program code is executed from beginning to end, like a script. The programmer has the choice of defining functions, classes, etc., but does not have to. Programs can be merely a sequence of statements.
Entry point code is also executed once from beginning to end, however its intended use is within the defined entry points. So, while you can include statements, just like in a program, it would be problematic if they had any side effects other than setting up the entry points. To understand why, see below.
Programs get they own unique global scope every time they are executed, which is discarded when the program ends. In the case of configuration scripts, this is rather trivial: those scripts are all run in a single thread, once, and thus always use a single global scope. However, in a threaded environment—template resources—you may have many of these global scopes existing at the same for your the same code. To share state between the threads, you would thus need to use application.globals or similar mechanisms.
Entry point code works very differently: the code always uses a single global scope shared by all threads, and indeed the code is executed from beginning to end only when initialized. However, if a recompilation is triggered (say, if the source code has been modified), then a new global scope will be created, and the code will be executed from beginning to end again. The implications of this are discussed above in application.globals vs. global variables.
As you may conclude from the above, in threaded environments entry point code is executed from beginning to end much less frequently than program code. This means that if performance is crucial, you should prefer to use entry points over programs. However, there are usually a lot of other factors involved, and indeed the differences between these two execution environment types may not be the important once. For example, even though manual resources are implemented as entry points, there will likely be no performance advantage in using them instead of template resources, even though the latter involves a new global scope per each request. Creating a global scopes is very lightweight for most programming language engines: the differences in performance depend more strongly on what kind of code you chose to execute in the program.
The only Prudence feature that offers a clear choice between these two types is background tasks spawned via API. There, you can decide as to whether you want to use a program or an entry point.
Comparison Table
Feature | Type |
Configuration scripts | programs |
Manual resources | entry points |
Template resources | programs |
Background tasks | programs or entry points |
Filters | entry points |
Cache key template plugins | entry points |
Scriptlet plugins | entry points |
The Prudence Manual is provided for you under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The complete manual is available for download as a PDF.