Creating self-contained, executable Jars with Gradle and Shadow
Earlier this year, we started implementing a new backend REST architecture for our application based on Yammer/Coda Hale’s Dropwizard Framework. Since we don’t use Maven anywhere in our stack, we decided to use Gradle as our build tool. This gives us enormous flexibility as we built our new backend around the concept of “micro-services” – each service is isolated from the others and responsible for a specific piece of functionality.
Since these services aren’t WARs, we wanted a way to deploy a pre-packaged jar that we could simply execute on our servers. Maven has a nice tool for this – Apache Maven Shade Plugin. It repackages all of your application’s dependencies into a single JAR file and offers a variety of extension points for effecting the contents of the resulting file. Unfortunately, at the time, Gradle lacked a similar plugin (there are a variety of similar plugins but all had severe limitations for our application). Instead, we wrote a Gradle port of the Shade plugin – Gradle Shadow Plugin.
Shadow is a port of the Shade plugin including most of its extensions points. This makes it easier for converting existing Maven builds to Gradle. It is open sourced under the ALv2 and contributions are most welcome.
To Shadow enable your project, add the BinTray repository and the Shadow plugin dependency to your buildscript configuration like so:
By default, Shadow will bundle all our your compile and runtime dependencies into an additional JAR with the classifier ‘shadow’ appended to it. This JAR is directly executable by executing the JAR:
$ java -jar build/libs/shadow-blog-0.1-shadow.jar
Hello World
Notice that trying to execute the output of the normal JAR tasks results in an error due to the Groovy library not being available on the classpath:
$ java -jar build/libs/shadow-blog-0.1.jar
Exception in thread "main" java.lang.NoClassDefFoundError: groovy/lang/GroovyObject at
java.lang.ClassLoader.defineClass1(Native Method) at
java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) at
java.lang.ClassLoader.defineClass(ClassLoader.java:615) at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) at
java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at
java.net.URLClassLoader.access$000(URLClassLoader.java:58) at
java.net.URLClassLoader$1.run(URLClassLoader.java:197) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject at
java.net.URLClassLoader$1.run(URLClassLoader.java:202) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
This simple example can be found on Github – Shadow Blog.
For more advanced usages of the Shadow plugin, refer to the README
Hi John,
Nice writeup! Could you tell me which limitations you ran into? I did the same thing; creating an executable jar using Gradle for my dropwizard project, and after some initial hickups, it works fine:
https://github.com/bodiam/epub-organizer/blob/master/epub-organizer-dropwizard/epub-organizer-dropwizard.gradle
Erik
Hey thanks man that helped quite a bit. I got my application to work with it.
Erik, I don’t think the jar task in gradle will include all the dependencies so you can’t run it standalone. It seems other folks were using the fatjar plugin to solve this problem – https://github.com/musketyr/gradle-fatjar-plugin . I wonder how shadow improves on fatjar. ( Dropwizard with fatjar — https://github.com/tomaslin/dropwizard-gradle-groovy/blob/master/build.gradle )
Shadow has a larger feature set (ported from Maven’s shade plugin) and is MUCH faster than the fat-jar plugin. fat-jar was taking ~8 minutes for us to build and shadow builds in 20 or 30 seconds.
@Erik – we evaluated OneJar (https://github.com/rholder/gradle-one-jar) & FatJar before deciding to write Shadow. OneJar didn’t work for us because service decriptors didn’t work correctly and FatJar was both too slow & didn’t allow us to configure everything we wanted to.
@Tomas – Our biggest complaint against FatJar was speed. It was taking minutes to execute the fatjar task whereas Shadow is in the 20 second range for a simple build. FatJar does its work by physically decompressing the Jars to a staging directory and then re-jarring it. Shadow uses Jar I/O Streams to read from the resolved libraries and write directly to the output file – no intermediate staging. Additionally, Shadow is a port of Apache Shade which is the predominant tool in the Maven world for doing this (referenced by the Dropwizard documentation itself). We thought a port of Shade to Gradle would be helpful with lowering the entry bar of converting complex builds from Maven to Gradle.
Thank you. This seems to solve a problem loading static content as well.
This is wonderful! It would be great if you managed to make the plugin available in Maven Central to make it more accessible. (It should not be too complicated via Sonatype’s OSS offer).
Regards, Jakub
Jakub –
Thanks for taking a look. There is an outstanding issue in Github to add this feature as well.
I plan in the next version (0.8) to release Shadown in the JCenter repository.
JCenter is just as easily accessible from a Gradle build as MavenCentral and a bit easier for me to maintain and use.
In addition, there will likely be a shift in the artifact’s group name to more appropriately locate the plugin.
Thanks for your interest! If you have any other issues or suggestions when using Shadow, please raise them on the Project’s Github page.
John
Hi I have a question. I use gradle with plugin shadow and all works well when I use ‘gradle shadowJar’. I have problem when I publish my artifacts to maven, e.g. ‘gradle publishToMavenLocal’ – the uber-jar is not created – my jars are without dependencies.
How can I deal with it?
Love the look of the plugin.
Any further thoughts on publishing to Maven Central? Having the option (esp I’m circumstances where repos are limited) would be a massive plus.
Thanks for what looks to be a great plugin.
Love the plugin. Can’t use the script creation though.
In the case where someone needs to add to classpath (I do realize all JARs are supposed to be in the shadow jar, but….) in order to set a java agent (say jolokia) and have it found by the JVM, there is no way to do adequate JVM options and/or classpath modification.