Hosted by Three Crickets

Sincerity
Packaging and Bootstrapping
For the JVM




Please note that Sincerity is still under development and that this documentation is incomplete.

Sincerity logo: jackrabbit and flowers

Introduction

Sincerity is a tool for deploying, installing and bootstrapping software stacks on top of the JVM. It makes these tedious tasks easy, simple and fun.
From the user's perspective, Sincerity makes it easy to install complete products and stacks, or individual modules and libraries, into portable "containers," which are nothing more than straightforward file directories. According to your preferences and constraints, you can use either Sincerity's pretty GUI or the powerful CLI.
From the provider's perspective, Sincerity is distribution system: simply host your packages in a "repository" (a simple web site) and let Sincerity do the rest. Configuration and bootstrapping of applications is easily controlled via simple JavaScript code, and Sincerity's growing ecology of plugins makes it especially easy to add features such as centralized logging and robust daemons with surprisingly minimal fuss.
Sincerity was born of many years of experience writing complex software for the JVM. The rest of this chapter summarizes this experience and the problematic reality that made a tool such as Sincerity necessary. If this sounds dreary to you, feel free to skip to the tutorial for now, and come back here later!

Principles

After using used Sincerity for a while, you'll wonder how you could ever have lived without it.
Indeed, Sincerity arrives after years of us having to repeat the same development and deployment tasks over and over again for every new project: download, unzip, copy, rename and configure, hopefully while staying organized as to dependencies and versions. Solutions like Maven create their own problems: "enterprise"-style complexity, enormous XML configuration files, and a Java-centrism that is becoming a burden as more of us are developing for the JVM without Java.
We decided that enough was enough! Sincerity intends:
  1. To simplify and unify the installation of JVM applications, services and libraries. You should never have to download idiosyncratic distributions and read through pages of installation instructions. We aim for a simpler recipe.
  2. To simplify and unify deployment via the JVM. It seems that every application and service has its own set of bootstrapping scripts, service wrappers, logging configuration, directory structures, etc. We can't entirely smooth out the quirks, but we can make your deployment experience consistent.
  3. To be language agnostic. The JVM is no longer the exclusive domain of Java. A rich ecosystem of languages has grown around it, and Sincerity lets you manage installation and deployment without ever having to write or think in Java if you don't want to.
  4. To be culturally agnostic. The JVM is not the exclusive domain of enterprise applications. If you think and work like an agile technology startup, Sincerity is here for you.
  5. To cultivate an ecosystem. Three Crickets, the company behind Sincerity, maintains a collection of quality plugins that do all the above, and the list keeps growing. Developing plugins is a piece of cake and you're strongly encouraged to develop your own using the straightforward API. This Sincerity Manual contains everything you need to know to get started.

Lather, Rinse, Repeat

The free software and open source movements have utterly changed how we develop software.
Reusable libraries had existed freely before, but these movements have created a culture of sharing, fueled by viable business models, culminating in an unprecedented wealth of solutions. For any problem you encounter in your everyday development work there is likely a library out there to help you that you can download for free. "Your mileage may vary," as they say: quality may not always be up to snuff, and no warranty is provided, but the source code is included and you can make it better, for yourself and for others. Importantly, it's relatively future-proof to depend on free software: you can be certain that your license to use the library will not be revoked and that bugs could be solved, by you, by the community, or by hired help.
(You do need to worry whether the software breaks any owned patents, but that problem exists for any software, whether it's free or proprietary, from a third party or developed by you.)
This wealth of solutions also creates challenges. There are several packaging, versioning and delivery standards for libraries. And when it comes to platforms and frameworks, there is no standard way to deploy software on top of them. If your project is a composite of many of these, you will find yourself spending a lot of time making sense of these various schemes and integrating them into a system that is maintainable by you in the long run. And if your software is itself modular and redistributable, you will find yourself having to pick one of the many different methods, or inventing one of your own. So, when it comes down to it, while free software can save you a lot of time and effort in terms of development, you end up spending extra effort on integration and maintenance. Annoyingly, you'll find that much of this work is repetitive, unnecessarily so. If you're a programmer used to making code reusable, you'll may such repetitive work especially annoying.
When it comes to the JVM, a few products have been widely adopted that make some of this work easier. However, experience has shown them to have too small a scope: they solve very specific problems, but do not address the complete challenge. Additionally, since they are already a few years old, they predate the linguistic revolution that is happening in full force on the JVM: no longer is Java the only good choice for leveraging the platform. New and popular languages like Scala, Clojure and Groovy offer a new experience and culture, while Nashorn, Rhino, Jython, JRuby, Quercus and Luaj bring popular languages and their paradigms to the JVM. Indeed, since version 7, the JVM has added support an opcode (invokedynamic) that cannot be normally generated by compiling Java language code: for the first time in its history, the JVM is made for languages that aren't Java.
With that in mind, Sincerity is designed from the ground up with multilingual support, which means that not only is knowledge of Java code never required, but also that the culture of dynamic languages and their standards are intrinsically supported: you can include dependencies from Ruby gems, Python's PyPI repository and PHP's PEAR repository. Moreover, Sincerity has standard plugins that make installing and working with these dynamic languages especially easy.
You might want to jump straight to the tutorial to see how it works, but you're also invited to stay here and look at some of the development and deployment tasks that Sincerity tackles.

Dependency Management and "DLL hell"

How do you get a library working with your application? Let's see:
  1. Find the library's web site.
  2. Look for the "download" button.
  3. Download the latest version: note that you want to write down all versions of all libraries you are using, so that you can handle upgrades and possible conflicts.
  4. Open the distribution archive: you want to be organized about this, so that you can find licenses, documentation, etc., later on.
  5. You need to make put the jar in your classpath for the following environments:
    1. Your development environment: you might also want to link source code and documentation if they are available in the distribution.
    2. Your deployment environment: the application needs it to run, so you to need to somehow include the file in your bootstrapping script.
    3. Your distribution, assuming you are distributing your application: this is optional, since you might decide not to include this dependency, and to have the users download and install it themselves.
  6. There might be configuration files (property files, XML, etc.)
    1. You might need to make different versions of these for your different environments.
    2. The configuration files might not be flexible enough for how your application runs, with too much assumed or hardcoded, so you will need to either:
      1. Document this fact for the user to handle on their own.
      2. Generate the configuration files during your application's bootstrapping process.
      3. Patch the library to allow for the flexibility you require.
  7. Once in a while you want to check for upgrades, which might mean subscribing to an RSS feed or mailing list, or just reminding yourself to check the web site.
  8. The library might have requirements, so you need to make sure to do all the above for them.
The above steps involve a lot of work. And what if you have 20 dependencies?
This is not a new problem, and there are already a few solutions for it. Firstly, there is a straightforward standard for JVM repositories, iBiblio/Maven, which is widely used by many projects. But it requires you to use one of two tools: Ivy, which does a good job of downloading dependencies (and is used internally by Sincerity), but does nothing else, or Maven, which is a sophisticated, heavyweight project management tool with a steep learning curve, and which requires you to work entirely within its domain. We'll compare these tools in more depth to Sincerity later on, but for now let's just say that the former is too limited in scope, and the latter too constraining. There are also various difficulties in configuring these tools: Sincerity "just works," immediately and easily, and also handles bootstrapping and assists in configuration.
There's also the problem of being forced in the JVM bubble: if you're using Jython, JRuby or Quercus, then you have to also work with the repository standards of Python (PyPI), Ruby (gems) and PHP (PEAR). Sincerity is designed to support all of these standards.
Then there's the issue of potential conflicts, a.k.a. "DLL hell": What if one application you're working on requires one version of a specific library, and another application requires another? What if this happens within different parts of the same application? Again, there are standards and tools for this—OSGi and Jigsaw—but they require you to work entirely within the paradigms they enforce. Sincerity doesn't stop you from using them (in fact, it has great support for the Felix OSGi container), but definitely does not force you to play by any special rules. From the bottom up, Sincerity is designed to be as straightforward and universal as possible. See the detailed comparison to OSGi below for more information.

Bootstrapping

The JVM is packaged as a set of command line utilities, plus a few plugins for specialized environments. It does come with one simple way to distribute programs—executable JAR files—but that would only suffice for the most trivial programs.
For anything more complex, you will need to handle bootstrapping your application. This means, at the very least, finding the right JVM on the machine (more than one may be installed), and then loading the application via the "java" tool. Usually, however, it ends up being far more complicated: rummaging through environment variables, detecting the host operating system and environment in order to set specialized JVM flags and load optimized native libraries, and because this is so complex, you'll want to responding to specialized bootstrapping flags set by the user. Indeed, many JVM-based products won't "just run," but will in fact require you to set a host of environment variables first.
All this work happens before the JVM even starts. Thus, it's usually handled by writing a shell script, which is almost always immediately runnable. Depending on how many operating systems you want to support, this may mean, at the very least, writing one for *nix systems and one for Windows systems. This is highly specialized work, and a development project with its own challenges, so some projects choose to avoid scripts and develop native binaries that handle bootstrapping. And then there are installer products that purport to do this all for you.
And what if you want the software to run as a daemon, system service, or cron job?
And what if your software is not just one program, but also contains a set of tools that you also need to bootstrap?
The bottom line is that bootstrapping is very hard to get right, and there are many complicated approaches to it. It's a shame that so many JVM products keep trying to implement the same bootstrapping solutions from scratch. Sincerity streamlines this in two ways: first, by providing you with working shell scripts, and second, by having these scripts delegate the process as soon as possible to a JavaScript program running in the JVM. Once on the JVM, Sincerity offers a range of installable plugins that handle various configuration and deployment tasks, including running the software as a daemon.
Why JavaScript and not a different scripting language? We deal with the question in length below.
What this means is that most products won't have to do anything beyond what Sincerity offers out of the box, and those with specialized bootstrapping will be able to write portable JavaScript programs, instead of having to deal with complex shell scripting.

Configuration and "XML hell"

Between bootstrapping and reaching full usability, your product has to configure itself. Will you choose a properties file? XML? Something else? And where is the file located?
Well, consider that all the libraries you use had to make their own choices for configuration. A non-trivial JVM product could thus require several configuration files, in different locations, with different configuration rules.
But there's a more serious problem to most of these approaches: they are unnecessarily rigid and static. While there are many advantages to using text files for configuration, the choice of technologies is baffling. Possibly the worst choice is XML. This language, ostensibly a "markup" language, is marking up nothing when used for configuration: it's instead used as a cumbersome format for structured textual data. And it gets far, far worse: XML configuration files are often used programatically in the JVM world, to construct JVM classes and call JVM methods. The best known, worst offenders are Log4j and Jetty. There, XML is used as if it were a scripting language, the clumsiest you have ever seen.
The use of XML for configuration is part of what we call "XML hell," which refers to programmers being swamped with countless overly-verbose XML files. XML is also often abused as an interchange format on the Internet, and a descriptor format in much of the JVM enterprise industry. Enough already!
The excuse for this insanity, one would guess, is that the ability to parse XML is standard on many platforms, including the JVM. But, interpreting this XML is far more complicated that just parsing it. In essence, parsing a general-purpose XML for something like Jetty involves writing a complete (more likely, not complete enough) scripting language engine. Another excuse for "XML hell" could be part of the general over-enthusiasm with XML, and the untested faith that standardizing on a single format would lead to greater interoperability. Again, this is madness: unless you couple the XML file with the code that can make sense of it, the ability to parse them is of little use.
Another approach, better than XML, is to create a Domain-Specific Language (DSL). But DSLs require a lot of work, both by developers and by users who must learn them.
Sincerity is here to stop the madness: wherever possible, it standardizes on using JavaScript for configuration. (Why JavaScript? We deal with the question in length below.) With JavaScript you can instantiate objects, call methods, insert conditionals and loops using natural programming paradigms, instead of shoe-horning them into XML. At its simplest, a JavaScript configuration file can look identical to a simple properties file: straightforward assignments of values to configuration parameters. But, you also have the option of injecting interpreted code where appropriate. And, of course, it's still just text files that don't need to be compiled, and can even be picked up and re-interpreted at runtime, so you're still absolutely within the "configuration-by-text-file" paradigm.
If you've never tried the "configuration-by-script" approach before, you might be skeptical about its benefits or worried about the extra weight it adds. But Sincerity's JavaScript engine is very lightweight, and we're convinced that once you try this approach, you will avoid all others. For an instructive example, install Sincerity's logging plugin, and take a look at the logging configuration files. Now compare them to the "official" Log4j formats.
One consequence of this approach is that the line between bootstrapping and configuration gets blurred. They end up as one integrated phase: a bunch of JavaScript programs strapped together. This leads to both simplification and greater flexibility for you. This approach leads to exceptionally dynamic configuration systems that can adopt to any operating environment.
We really hope to see "configuration-by-script" used throughout the JVM world, even for projects that do not want to or cannot use Sincerity.

Logging

The JVM has a few good, widely-used logging APIs, as well as a great glue library—SLF4J—that can bridge between them. But there's quite a bit of work involved in getting all these libraries working together. It seems that every JVM product has its own way of doing this. Logging is important, and can't be relegated to an afterthought: if it's not properly configured and well integrated, it's close to useless.
Sincerity takes logging very seriously: it provides a plugin that does much of the work for you, and extensions that further enhance logging. For example, one extension funnels all logs to a centralized MongoDB collection, perfect for distributed cloud deployments. And this system will work with practically any JVM library.
Note that logging configuration is handled via the "configuration-by-script" approach mentioned above, and is well integrated with the whole ecology of Sincerity plugins.

Why JavaScript?

There are many great scripting languages for the JVM. Note that by "scripting" here we mean that these languages are immediately runnable from the textual source code. This doesn't have to mean that they are "interpreted": many of these languages compile some or all of your code on-the-fly. So, why has Sincerity standardized on JavaScript, rather than Groovy, Scala, Clojure, Lua, Python, Ruby or others?
There's no single answer, but rather a combination of factors that make JavaScript attractive:
  1. JavaScript is very well known. Since its original introduction into web browsers, and universal adoption as a web standard, it has gained an enormous skill share in the industry. There's a wealth of education material available for it.
  2. Its implementations are relatively lightweight, in that JavaScript is both fairly minimal linguistically, and also does not have anything like a standard library, of the kind you would find in Python and Ruby. This allows Sincerity to have a much smaller footprint than if it were to use Jython, JRuby or even Groovy. Note that not having a standard library can also be seen as a disadvantage, but in the case of Sincerity it's dealt with in two ways:
    1. Since we are running JavaScript on the JVM, we have full access to the Java standard library.
    2. Sincerity comes with the Sincerity JavaScript Library a very lightweight framework that makes working with JavaScript on the JVM a little bit easier.
  3. JavaScript is very future-proof: not only is it an open standard (where it's called "ECMAScript"), but it is baked into the JVM (from version 8) as the Nashorn engine.
  4. JavaScript is actually a nice language. It has been the target of a lot of negativity from programmers who had to work with it in browser environments, but we believe the fault is more of the environment (the browser DOM's poor API and many annoying differences between various browser implementations of it) than the language itself. It encourages prototype-oriented programming, which can easily emulate object-oriented programming, as well as other paradigms. In fact, JavaScript shares much of its scoping and function handling with Scheme, a language that is generally admired. You can think of it this way: JavaScript is Scheme with a C-like syntax.
Despite these general advantages, you might still prefer to use another scripting language for your own work. Luckily, Sincerity, with the help of the Scripturian library, will let you write plugins in Python, Ruby, PHP, Lua, Groovy or Clojure. The only disadvantage is that you would have to include the appropriate language engine as a dependency. In the interest of keeping Sincerity and its ecology of plugins lean and mean, we want to encourage the use of JavaScript for plugins that are intended to be shared with the community.
Just to be 100% clear: this preference for JavaScript only applies to Sincerity plugins, configuration scripts, and skeletons: you are definitely welcome to write your application in whatever language you choose. In fact, Sincerity contains great plugins for many popular JVM languages, as well as skeletons for complete language-specific frameworks, such as Django and Rails.

JavaScript vs. Shell Scripting

This section is meant for those of you who are comfortable with shell scripting, and are wary about Sincerity's use of JavaScript for bootstrapping.
  1. You might think that shell scripting would always be more portable than a scripting language running inside the JVM. But, think again: the point of your bootstrapping work is to get into the JVM, in order to run your application. If that doesn't work, then your whole application won't run, and portability is moot. Sincerity does have shell scripts, but they're designed to delegate to the JVM as soon as they can.
  2. You might be concerned about startup delay: starting up the JVM with all the JavaScript engine classes is much slower than starting up a shell script. This is true, no doubt, but since version 7 the JVM is doing better. Also consider that you have to get into the JVM anyway for your application to do anything useful. Still, if your application has a lot of tools that do not always require the JVM, and would be adversely affected by the JavaScript bootstrap times, then by all means write them as shell scripts! You can use all of Sincerity's other features when you need them.
  3. Shell scripts treat most of your program as an opaque, black box. But with JavaScript running in the JVM you can call parts of your API before the application truly starts. This can allow for much more powerful, dynamic bootstrapping.
  4. JavaScript is likely richer than your shell language. Sure, bash 4.0 and PowerShell are a leap forward compared to what we had 20 years ago, but they're still quite constricting.

Comparisons with Other Solutions

Sincerity vs. Maven

Apache Mavenis a comprehensive solution for managing Java projects, handling building, dependency management and distribution. It contrastingly combines a lot flexibility on the one hand—an open plugin API built on the Plexus IoC container—with deliberate rigidity on the other hand: a strict reactor-based, multi-phase cycle. In particular, Maven's design goes to great lengths to keep you from affecting the order of operations: you are supposed to configure your project, and let Maven decide what to do when. For those used to scripting their build process, this approach may initially seem baffling and restricting. However, there are significant benefits to this approach when working with very large, complex projects: instead of coding and maintaining nightmarishly long build scripts based on dozens of changing environment variables, you can sit back and let Maven analyze the entire operation and then do the right thing.
But, for this to work, you need to play by Maven's rules, and that's where things get tricky. Small deviations from the strict assumptions Maven makes throw you down the rabbit hole of plugins and hacks, as you struggle to shoe-horn a simple procedure into a product that abhors procedure. Specifically, Maven's ideal environment is one in which your versioned modules are written in Java mapped to single jar files. Anything even slightly different becomes painful and hacky.
Both Sincerity and Maven handle downloading dependencies, but other than this apparent overlap these products have different goals and scope. Importantly, they can be very complementary. One way to think of this is that Maven could come first and Sincerity come second: Maven could help you build your project and repositories, while Sincerity would handle your deployment container. Maven won't help you run your application: its output is jars of compiled code, source code or documentation, and it doesn't handle their bootstrapping or runtime configuration. On the other hand, Sincerity does not build your project, nor does it make any assumptions about how its built: you can use Maven, Ant or anything else.

Sincerity vs. OSGi

An "interface" in the JVM lets you create a standard protocol, such that you can plug in various implementations of it—"classes," with "methods" as the entry points—at runtime. The protocol is enforced by the JVM, which will not let you plug in implementations that do not fit the interface. OSGi takes this up a level, by providing a much broader concept of "implementation." The implementation is a "bundle" that can contain any number of classes.
So far so good, but it gets complicated fast. OSGi takes it up one level more: the protocols are published and endorsed by a community of providers, with the idea that different providers (software vendors or departments in a large enterprise) can provide bundles to implement them, which would all work together perfectly. With this broader ambition, "DLL hell" suddenly becomes a far more malevolent enemy: bundles are often black boxes that you cannot easily patch to use a shared version of a dependency. There's thus a real need for a standard solution of runtime code compartmentalization, which OSGi provides via a clever system of classloaders.
…Which, of course, introduces its own set of problems. To get its classloading scheme to work, OSGi requires strict separation of classloading between bundles, which in turn adds subtle and mischievous restrictions to your usual JVM work. This is not entirely bad: working within these limitations does encourage clean, sharp boundaries between your modules, and goes a long way towards reducing classloading confusion. It's not, however, trivial by any means, and all your bundles must be designed with this in mind for OSGi to work properly.
One very useful side effect of having the framework control classloading is that entire bundles can be loaded and unloaded during runtime. Indeed, OSGi defines protocols for starting, stopping and hotswapping services. This is a powerful feature in itself, and is indeed the entire motivation for using OSGi in some cases. (Though, if that's your reason, you might want to look at other, simpler ways to enable hotswapping, rather than embracing the whole of OSGi.)
It's worth noting, however, that there is a more straightforward solution to the problem of "DLL hell": Why not run each "bundle" as a separate process? Each JVM would load its own classes as necessary, and never will they mix or conflict. This makes a lot of sense if you're running a distributed system, since you're already dividing your software among many machines and processes, and indeed many parts of your application may not be JVM-based at all, and can't be run in a single process anyway. As for starting and stopping your "bundles," the operating system already does a good job of managing processes, so you don't need OSGi's protocol for that. From this perspective, you can see that OSGi is, in effect, creating a virtual operating system inside the JVM, where "bundles" are very much like operating system processes.
Indeed, the original target environment for OSGi was precisely one in which all bundles ran in a single process, in shared memory space: it is the world of embedded computing, where the runtime is variously confined, such that you are either limited to a single process due to limited resources or security concerns. In such environments OSGi may be your only good solution for the problem of modularity and pluggable services. Still, OSGi has also proved popular in large enterprise environments, where it allows for modules to be treated more abstractly whether or not they are running in a single process.
Sincerity, in itself, takes the more straightforward approach: such high-level modularity is provided through the notion of "containers," which you can easily create, clone and change, and start and stop as processes, specifically the service plugin makes it especially easy to run them as daemons and services. Containers can then talk to each other (and to other services) using whatever technology is appropriate, be it REST, SOAP, message queuing, Hazelcast, etc. That said, OSGi may indeed be appropriate for your project, and Sincerity provides a nice Felix plugin to get you up and running. The point being that Sincerity was designed to be neutral to the technology of modularity, introducing no special restrictions for users that do not need them.

The Sincerity Manual is provided for you under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The complete manual is available for download as a PDF.

Download manual as PDF Creative Commons License