Diligence is still under development and incomplete, and some this documentation is wrong. For a more comprehensive but experimental version download the Savory Framework, which was the preview release of Diligence.
Progress Service
If you've read Prudence's Scaling Tips article, you know that for potentially long-running tasks you want to release web request threads as soon as possible, and notify the user in some way as to when the task is finished. This service helps you do exactly that.
For a use case example, consider an application that searches for flight information using several databases and services. The search can take many seconds, if not minutes! Of course, you do not want to hold up a web request thread and have the browser spin while the search is going on, so you turn to Diligence's Progress Service.
It works like this: you create a "process," which is stored in a MongoDB document, and you can asynchronously mark when certain "milestones" are completed, including the final completion of the whole process. Processes can be associated with a user, which allows you to use the authorization service to allow only that user access to the process' status, and also to allow the user to query all processes associated with them.
The service supports two ways of letting the user know the status of the process. The first is for short-term processes: a drop-in fragment that simply shows the current status of the process and uses browser JavaScript to refresh the page every few seconds. The user would see milestones along the way to completion, if there are any, and eventually be redirected to another page when the process completes (or fails!).
For longer running processes, you cannot expect the user to wait in front of the web browsers. In these cases, the Progress Service uses the notification service to notify the user about milestones, success and failure. Additionally, we provide a drop-in fragment that would allow the user to see the current state of the process on the web, and another one that lets the user access all processes associated with them.
Usage
Make sure to check out the API documentation for Diligence.Progress.
Trivial Example
This fake process will simply do nothing until its expiration:
document.executeOnce('/diligence/service/progress/')
var process = Diligence.Progress.startProcess({
description: 'Searching for your flights...',
maxDuration: 20 1000,
redirect: conversation.reference
})
process.redirectWait(conversation, application)
That final redirectWait call will send the user to a "please wait" page which will show "Searching for your flights…" as the text, and have a progress bar. The page will automatically refresh and show ongoing progress. After 20 seconds of this, it will redirect back to this page. Note that you can specify different redirect URIs for success, error, timeouts, etc.
The "please wait" page is in "/diligence/service/progress/wait/". If you don't have it in your "/fragments/" then a default page will be used, which is in your container's "/libraries/prudence/" directory. You can use that as a template for your own custom page.
Example with Milestones
You can launch a task from within startProcess, which in turns call the Prudence.Tasks API:
var searchString = 'flight #1234'
var process = Diligence.Progress.startProcess({
description: 'Searching for your flights...',
maxDuration: 60 1000,
redirect: '/flight/results/',
task: {
name: '/flight/search/',
searchString: searchString, // this is our custom field
distributed: true
}
})
Our "/libraries/flights/search.js" would look like this:
document.executeOnce('/diligence/service/processing/')
var process = Diligence.Progress.getProcess()
if (process && process.isActive()) {
var task = process.getTask()
var milestone = process.getLastMilestone()
switch (milestone.name) {
case 'started':
process.addMilestone({name: 'ours', description: 'Searching our flight database'})
var found = searchOurDatabase(task.searchString)
if (found) {
process.addMilestone({name: 'done'})
} else {
Prudence.Tasks.task(task)
}
break
case 'ours':
process.addMilestone({name: 'partners', description: 'Searching our partner databases'}
var found = searchPartnerDatabases(task.searchString)
if (found) {
process.addMilestone({name: 'done'})
} else {
process.addMilestone({name: 'failed'})
}
break
}
}
Notes:
- The "Diligence.Progress.getProcess()" API works here only because we launched the task from within startProcess. (It works by putting the process ID in the task context.)
- The first milestone is always "started", and the last one is always "done". The name "failed" is reserved for failed processes, and like "done" will mark the process as inactive. Otherwise, you can set any milestone name you wish.
- You'll also see that we've handled each milestone as a new execution of the task. "process.getTask()" returns a copy of the arguments sent to the last Prudence.Tasks.task call, so we can simply call it again with the same arguments.
- Breaking up our work into separate tasks allows for better concurrency: we're not holding on the thread at once longer than makes sense. Also note that if the task is distributed, each milestone could be executed in a different node in the cluster.
- This method and also makes sure that a milestone will not be executed if a process expires (isActive would return false).
Reattempts
A common use case for the processing service is in dealing with an unreliable action that might actually succeed after a few attempts. You'd thus want to let the user wait until a certain maximum duration, and keep retrying every few seconds in the background until the action succeeds.
The Progress Service automates much of this using the "maxAttempts" key in "task":
var ipAddressOfRemoteLocation = '1.2.3.4'
var process = Diligence.Progress.startProcess({
description: 'Attemping to connect you to remote location {0}...'.cast(ipAddressOfRemoteLocation),
maxDuration: 5 60 1000,
redirect: '/remote/connected/',
task: {
name: '/remote/connect/',
maxAttempts: 10, // for reattempts
delay: 5000, // between reattempts
remoteLocation: ipAddressOfRemoteLocation // this is our custom field
}
})
Our "/libraries/remote/connect.js" would look something like this:
document.executeOnce('/diligence/service/progress/')
var process = Diligence.Progress.getProcess()
if (process) {
process.attempt(function(process) {
document.executeOnce('/mylibrary/connections/')
return connectRemote(process.getTask().remoteLocation)
})
}
Notes:
- The process.attempt call doest most of the work: it makes sure to call the task again if there's still time before the process expires and the maximum number of attempts has not been exceeded, waiting the appropriate delay before each attempt. Your function just has to make sure to return true if the attempt has succeeded.
- Each attempt will get a milestone name in the form of "attempt #X" where X starts at 1.
- If the maximum number of attempts has been reached, the milestone will be set to "failed".
- Reattempts are logged, to help you debug problems.
The Diligence Manual is provided for you under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The complete manual is available for download as a PDF.
